Software Factories
GAT, DSL Tools and DSD designers
If you're extending Visual Studio using the
VSSDK, you probably know that it's no trivial task. If you were thinking about doing some VS extensibility stuff but it all seemed a daunting task, look no further.
Both the experienced as well as the novice VS extensibility developers will find the
VSIPFactory a worthwhile addition on their toolbox. What's even more valuable, it comes with complete source code!
Just head over to the
homepage, it's filled with features and screenshots that should be enough to get you more than excited :)
Pablo (the project owner) is a hardcore developer who has been doing VS extensibility for years. He's the man behind the
Web Service Software Factory and some features of the
Software Factories Toolkit.
His blog is full of gems on software factory development and VS extensibility.
I've just now noticed that Ron Jacobs has uploaded our talk in TechEd Barcelona back last November.
It was supposed to be about designing orientation aware and DPI aware mobile applications using the Mobile Client Software Factory and its Orientation Aware Control Block (see http://orientationaware.net/ ;)), but ended up being about other parts of the factory, about ObjectBuilder and optimizations I did to avoid runtime reflection to inject dependencies, CAB, mobile development in general, etc. It was a fun interview :).
Go ahead and download it!
Software factories is an elusive concept nowadays. In this article, I will try to help you understand which technologies can be used today to build production-quality factories and the reasons for the perceived inconsistency in the tools and the vision on Microsoft's part.
None of what follows is speculation: I've been a main developer on one of the two core technologies that realize today' software factories (patterns & practices Guidance Automation Toolkit or GAT), and as such I've been in close contact with the teams and architects building the technologies I will explain here (including Jack Greenfield, Wojtek Kozaczynski, Mauro Regio and others). Also, my company is a leader in software factory development, providing most of the people that build the factories coming out of the most active practitioner in the area today: the Microsoft patterns & practices group. We also have several customers around the world that we helped reap the amazing productivity boost that today's factory tools can bring.
First and foremost: this article is NOT about the grand vision (still underway, rest assured, but nevertheless in the realm of the future) outlined in The Book: Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools by Jack Greenfield, Keith Short et all. I will talk about technologies that are ready for production today. This also means that the strict concept itself of a SF outlined in the book needs to relaxed for what's currently possible with the available tools. You can think of today's factories as infants that will grow with features as times goes by and the end-to-end SF vision becomes a reality. That doesn't prevent the "infant" from giving you a big boost in terms of productivity and software quality.
You can build SF today by using a combination of the following two tools:
-
DSL Tools: or Domain Specific Language Tools. This is a toolkit for creating VS-integrated graphical notations for a given model that you basically make up. It allows you to construct a custom-tailored designer for a model for your problem domain.
-
GAT/GAX: or Guidance Automation Toolkit/Extensions. This is a toolkit for extending VS with solution, project and item templates, code snippets and so-called recipes (or developer use cases) exposed as context menus within VS, that guide the developer in completing a complex task.
The problem is that integrating the two is not a trivial task. Why you may ask? Aren't they both coming from Microsoft? And how does GAT/GAX fit in the SF vision? After all it's not even mentioned in The Book!
Glad you asked ;). In order to understand this "mess", and figure out how to make it work for you, you need to understand a little bit of the background behind the two technologies. The DSL Tools is basically the first step in the road to realizing the full SF vision outlined in The Book. Hence, if you try to build a real-world factory just using DSLs, you will find it fairly incomplete and not supporting typical scenarios such as providing an initial solution structure where you will put your DSLs, etc. The missing features, though, are most probably those that will come (or become unnecessary) as the full vision is implemented. In the meantime, it feels lacking.
On the other hand, Microsoft patterns & practices (p&p) is NOT about building a compelling but somewhat distant feature, but about solving customers' problems today. p&p follows a process called "Customer Connected Engineering", which means customers get to influence what we do. Delivering guidance on how to best use the platform has been p&p traditional focus, but as guidance became more and more complex, thicker books and more complex libraries (think CAB and EntLib) just didn't cut it anymore. Customers were not getting the guidance in an easy to consume way. Hence, we set out to find a way to bring the guidance into Visual Studio, in a way that was easy to author (not only for p&p but for other companies delivering guidance internally) and contextual. The readily available VS extensibility mechanisms (add-ins and VSIP packages) did not satisfy either requirement, so we built our own framework on top of VSIP to allow for easier extensibility. And GAX was born. As the name implies, it's a mechanism for automating the guidance that you would have to otherwise perform manually by following a book's instructions.
You may notice that p&p software factories do not contain DSLs. This is in part because the goals of p&p SF are somewhat different than The Book SF. Their main purpose is to guide the developer through the best practices for implementing a given solution. In the end, it's the developer himself who implements the solution. It just so happens that he does so much faster and in a predictable way because he uses automated guidance within VS that generates code, unfolds boilerplate projects and solutions, provides code snippets, etc.
Given the different origins and goals of the two technologies, it may seem that they are hopelessly irreconcilable. Fortunately, that's not necessarily the case, and there are many ways in which they can be integrated. My partner Victor Garcia Aprea and Mauro Regio have published an article on MSDN about integration scenarios that include integrating the two.
A number of integration points are particularly critical:
- An initial solution structure is generally required: this can be done with GAX.
- Typically at a specific location in the solution, a domain model/designer could be added to design some area of the application (i.e. a business entities domain model, that generates business entities + data transfer objects + data access layer code + SQL files with stored procs for the DB). This needs to be done with a combination of DSL and GAX if you want the DSL to only be available at a specific location in the solution. (remember, GAX is about context)
- With the DSL in place, you may want to generate multiple code files (as in the case of the business entities model I mentioned above), maybe even in different projects, and maybe you even need to generate the projects themselves. Again, DSL tools alone is not enough. By default, it will do two things that are pretty bad for a real-world factory:
- The code (a T4 template) that transforms the domain model in a designer into actual code for your project is expected to live in the same project as the domain model file. You are given a custom tool that runs the transformation, and a solution explorer toolbar icon to run all the transformations in the solution/project ("Transform All Templates"). What kind of guidance are you giving your developers if they can modify it at will just by modifying the T4 template that generates their code? It's more like giving them an example that they can tweak, and typically you don't want your developers modifying the templates for code generation that your architects together with your senior devs have created to ensure quality code.
- Each T4 template results in exactly one big file with the output of running the codegen against the domain model. If your model designs business entities, you will end up with a huge file with all the entities in the model. Generally not a good idea, and certainly goes against the typical .NET coding practice of using one file per class.
And it will not do the other two things that you need: generate new projects, and generate files that target multiple projects. You can supplement the DSL with GAX in order to achieve all of this. Achieving the above two goals requires deep understanding of how both technologies work at a very granular level, though, but it's definitely possible (more on this later).
- You may want to launch recipes/execute actions from specific shapes in the domain model diagram (i.e. "Configure Security for Entity" or something like that, maybe even regenerate code for a specific model entity and not all of them). You can hook GAX recipes to the context menus on DSL diagrams pretty easily, and with some more work you can make it contextual enough to provide very granular visibility (i.e. only model entities that have property Foo = true).
Of course, the integration is not as seamless as it could have been. But I hope I shed some light on the current situation by sharing the background and how the two tools came to be. Moving forward, you can expect the experience to become much better, as the scenarios that led to GAT/GAX are integrated into the SF toolkit.
As you might have guessed, we (Clarius) have gone over each and every hop in this integration that you can possibly imagine. We have recently released a CTP of our Software Factories Toolkit, which helps integrate the two technologies, and which you can download for free.
The reality of mobile development (specially if you're creating a product) is not so different to that of web development: the clients that will be using the application may not be under your control, and they may support different sets of features and capabilities. In the ASP.NET world, this was solved by the set of mobile-enabled controls, that basically rendered different markup depending on the device. Even in ASP.NET v2.0, this solution (the so-called control adapters architecture) is still in its infancy and not fully completed in its integration with the "standard" controls.
In the mobile development world, the problem is not even in its infancy. Up to know, that is ;-).
In the devices world, the critical difference between clients is not on the platform (as in the web, where you have different browsers supporting different standards, etc.). When you decide to develop your application using .NET Compact Framework (CF), you're already filtering the range of devices to Windows-based (most probably Windows Mobile 5.0 if it's a brand new one) Pocket PCs or SmartPhones. In this scenario, your most challenging issue is how to support the different so-called form factors (or resolutions, such as VGA, QVGA) and screen orientations (can be landscape or portrait).
The Microsoft .NET Compact Framework (CF) 2.0 introduces some features that certainly help alleviate the problem, specifically in the WinForms area, where you can use the familiar docking and anchoring funcionality to have your controls automatically resize according to screen size. However, this falls short for most applications. For example, say you’re displaying a purchase order in a vertical QVGA screen (240x320). Most probably you have laid out the controls so that each value you want to show is displayed in a single “line”. Well, if the screen is rotated, you probably don't want to simply make your fields wider. That may be a waste of valuable space, and result in a poorer user experience by the requirement to constantly scroll the screen. In this case, it may actually be better to lay out controls differently, so that you can for example display two values per "line".
Of course, the logic of the user control/form (hereafter simply referred to as "control", which is even technically correct as the Form class inherits from Control too ;-)) is the same: the data binding, event wiring, the controller/presenter logic, etc. So in an ideal world the layouts would be sort of "skins" for the control, that are applied dynamically depending on the current device form factor and orientation.
If you think for a moment on the problem at hand, and how the ideal solution would look like, you may realize that the .NET Framework (and therefore the CF too) already provides a mechanism that allows applying "skins" to WinForms controls: localization. Making a form localizable allows you to completely re-arrange the layout for each supported language, including size, location, text, etc. Basically any property that has been "tagged" with [Localizable] can participate in this "skinning".
Good news is that the Mobile Client Software Factory (MC-SF) from p&p makes this ideal solution a reality with the introduction of a custom user control that you can use as the base class for yours. I've been working for the past few weeks on this feature, and I think it's an absolutely revolutionary simplification of the way you will author UIs for multiple devices. In future posts I will discuss its features and implementation, but in the meantime, here are a number of screenshots that should speak for themselves ;-)
In this picture you can see the new Rotate verb on the context menu of a derived control, and also the new dependent file TeamView.resources, which I'll explain in future posts and is the key to the multiple resources design. You can also see the new Orientation property in the Design category, as well as the Localizable property which is now read-only and forced to True. I already hinted that our solution leverages the localization features of .NET, and therefore relies on that property being True.
Here you can see that the horizontal layout for the very same control has a completely different design. The labels and textboxes are now on the right side, to take advantage of the extra space available in landscape mode, and to avoid unnecessary scrolling.
This is a screenshot of the emulator running the application with the orientation aware control. In a future post I will link to the download of this sample, so that you get to know the team behind the p&p Mobile Client Software Factory ;-) (that's what the data on the list is).
Finally, you can see that when the user rotates the screen, the control automatically re-applies the corresponding layout. Pretty cool, right? ;-) Stay tunned.
As annouced a while back, patterns & practices is coming with guidance and tooling on how to best author applications for mobile devices (Pocket PCs), initially targetting the Windows Mobile 5.0.
I’m very excited to be part of this project, as it represents a new challenge in terms of a new platform, but where most of the same sound architectural principles that we been applying over the past years working with p&p still apply.
What’s more, as you already know from the latest p&p deliverables, we’ll continue to be very agile and responsive to community feedback. For that very reason, we have already started releasing weekly drops for you to take a look at! Go ahead and subscribe to the workspace, and don’t hessitate to send your comments through the message board. The entire team is listening.
In my next couple posts I’ll talk about a cool feature for the factory that we’ve been working on for the past couple weeks: the Dynamic Resolution Control. In short: author your single application, give your custom controls multiple alternative layouts for the various resolutions (i.e. QVGA horizontal, VGA vertical, VGA Square vertical, etc.), and have the control dynamically re-apply the layout when the user rotates the screen! Stay tunned. The user experience in VS is pretty cool, I think :)
Say you have a custom DSL file in a solution. Say you want to process the model on that file using GAT or an AddIn or even your custom application code at runtime. You need to deserialize that XML (yes, it’s all XML in the end) into an instance of your domain model (root).
You will need two pieces of information: the .NET type of the domain model (that is, the parent of your domain model root concept) and its the corresponding designer. The following code does the trick, allowing you to deserialize a model and return the first instance of the given domain model type (i.e. the model root):
public static TRootConcept GetModelRoot<TRootConcept>(string modelFile, Type modelType, Type designerType)
{
// Create a new store to deserialize the instance to.
Store newStore = new Store();
Type[] metaTypes = new Type[] {
typeof(Microsoft.VisualStudio.Modeling.Diagrams.CoreDesignSurface),
typeof(Microsoft.VisualStudio.Modeling.Utilities.UtilitiesModel),
modelType,
designerType };
// Load these types into the store, so that it knows about them for deserialization
newStore.LoadMetaModels(metaTypes);
foreach (Type subStoreType in metaTypes)
{
// TODO: this will not be required in RC+ versions of DSL tools.
Activator.CreateInstance(subStoreType, newStore);
}
// Be version resilient
int majorVersion;
int minorVersion;
GetModelVersion(modelFile, out majorVersion, out minorVersion);
// Deserialize the file into the store
using (Stream fileStream = File.OpenRead(modelFile))
{
XmlSerialization.DeserializeStore(newStore, fileStream, majorVersion, minorVersion, null, null);
}
// Locate the attribute that will give you the Guid of the element to find
MetaObjectAttribute metaObject = Generics.GetCustomAttribute<MetaObjectAttribute>(typeof(TRootConcept));
if (metaObject == null)
{
throw new ArgumentException(String.Format(
CultureInfo.CurrentCulture,
Properties.Resources.MetaInformationForElementNotFound,
elementType));
}
// Return the first one we find
foreach (object element in newStore.ElementDirectory.GetElements(metaObject.Id))
{
return element;
}
return null;
}
The GetModelVersion helper method simply reads the major/minor values from the XML itself:
private static void GetModelVersion(string modelFile, out int majorVersion, out int minorVersion)
{
// Get versions from model:
//<om:MetaModel MajorVersion="1" MinorVersion="1" xmlns:om="http://Microsoft.VisualStudio.Modeling">
using (XmlReader reader = XmlReader.Create(modelFile))
{
reader.MoveToContent();
if (reader.LocalName != "MetaModel")
{
throw new InvalidOperationException(Properties.Resources.FileDoesNotContainModel);
}
majorVersion = Int32.Parse(reader.GetAttribute("MajorVersion"), CultureInfo.CurrentCulture);
minorVersion = Int32.Parse(reader.GetAttribute("MinorVersion"), CultureInfo.CurrentCulture);
}
}
Update: here's how to do it with the June 06 bits.
