2008-09-11

A Python library for UML

While redesigning pyswarmSDK, I came to the decision to move all UML-related parts from the pyswarmSDK code-base into a separate library for UML models. Such a lib could be used by Python tools, that need to work with UML models, or to be more precise, to work with the data in the UML model.

This lib, pyswarmU2L (U2L = abbr., UML 2.x Library), will be able to import an UML model that is serialized in any supported XMI format, to create a new UML model or to manipulate an existing UML model, and to export an UML model to any supported XMI format.

Each element in an UML model is an instance of a metaclass specified in the UML meta-model. The meta-model used by this lib is based on specifications published by the Object Management Group (OMG), particularly the OMG Unified Modeling Language (OMG UML), Superstructure, V2.1.2 (PDF, 5.8MB). Please note that no support of diagram exchange is planned yet for this lib.

Creating UML model elements
For each metaclass the lib will provide a method to create a new instance of this metaclass. So any client can create its own new UML model from the scratch.

pyswarmU2L.newMetaclassName([propertyname=value,..])


Where MetaclassName is a placeholder for the class of the UML metaclass. As parameters the client can use name-value pairs for properties that the meta model specifies for this UML metaclass.

BTW, whenever I use the term "metaclass" in this post, I refer to metaclass as used in the UML meta-model, not as used for the metaclass concept of Python. I don't think that Python's metaclass concept will be used in the implementation of this lib.

Example:
import pyswarmU2L
# create new Model instance and new Package instance:


sales = pyswarmU2L.newModel(name='SalesApp')

# is instance of pyswarmU2L.meta.AuxiliaryConstructs.Models.Model

customers = pyswarmU2L.newPackage(name='CustomerManagement')

# is instance of pyswarmU2L.meta.Classes.Kernel.Package
Since the UML metaclasses Model and Package are subclasses of the abstract UML metaclass NamedElement, there is a property name with the type String inherited.

If shown in a diagram, the two instances in the UML model would look like this.
On the left side you see the Model instance with the property name="SalesApp", and on the right the Package instance with the property name="CustomerManagement".

Manipulating an UML model
Next, we want that CustomerManagement package is owned by the SalesApp model, so it will become part of the entire UML model. The meta-model defines that any Package instance can own other Package instances. The property name for this owned elements is "Package::nestedPackage"). Since Model is a subclass of Package, the SalesApp model can own the CustomerManagement package:

Example:
sales.addNestedPackage(customers)
Alternatively, we could instead use the exactly opposite role of this ownership relation, where the property name is "Package::nestingPackage". Since there must not be more than one owner for an Element, the setter method name on this direction of the relationship is beginning with "set".

Example:
customers.setNestingPackage(sales)
Okay, both ways have the same outcome, that the sales model owns the customers package. Visualized in UML diagram:
You also can get property values of any instance. In the case the property has a primitive type, you can get the value of the property by using:

ModelObject.getPropertyName([ClassifierName=value,..])

Where ModelObject is an instance of a metaclass and PropertyName is the name of the property of which you want to get the value. Of course you can only query values of properties that are available for the instantiated metaclass.

Example:
>>>print customers.getName()
'CustomerManagement'
If the type of the property is a metaclass with a multiplicity of 0..1 or 1..1, you will get one instance of the defined metaclass as return value or None if there is no instance.

Example:
>>>owner = customers.getNestingPackage()
>>>print owner
>>>print owner.getName() 'SalesApp'
If the multiplicity of the property exceeds an upper limit of 1 -- in these cases the UML meta-model usually defines 0..*-- the getter method for this property will return a collection of metaclass instances. Of course the collection can be empty if the lower limit is 0.

To exemplify collections, we add another package to the sales object, before iterating over the collection.

Example:
>>>orders = pyswarmU2L.newPackage()
>>>orders.setName('OrderManagement')
>>>sales.addNestedPackage(orders)
>>>for p in sales.getNestedPackages():
... print p.getName()
'CustomerManagement'
'OrderManagement'
Visualized in a diagram our UML model would look like this now:
Importing and exporting XMI
Following some examples how this UML model could be extended with elements from an existing XMI file, and how the entire UML model could be serialized into an XMI file.

Import an XMI file, either a string with the XML or an URL providing the location of the file, as following:

Example:
xmiData = ''' .....many more data here!..... ''' billingModel = pyswarmU2L.import(xmiData)
After importing an XMI file the data is available as UML model, according to the meta-model, and the client can work with it:

Example:
>>>print billingModel
>>>sales.addNestedPackage(billingModel)
After nesting the imported model into our existing model, the diagram would look like this:
Finally, we can convert the entire UML model to any XMI format that is supported by the lib:

Example:
newXmiData = sales.export(extender='FooUML', version='1.3')
By providing extender and version values, the lib is told which XMI format exactly to use. The export() method returns the corresponding XMI data as string and the client can do further processing of the string, i.e. write it to the file-system.

So far these are my considerations regarding a Python library for UML models. There are still some open issues. Especially the XMI filters will be tricky, since the differences between the formats and the meta model can be rather big.

Any comments or questions are welcome.

2008-04-14

Is it possible to generate use-cases in Python? (Part 1)

Recently I wrapped up some of my ideas how the graphical user-interfaces --that I want to be created by the pyswarm SDK-- may look like for end-users. I have focussed on the user experience with a desktop GUI and the built-in widgets (thanks to wxPython/wxWidgets). Furthermore there is also a draft about the planned support to create/generate also the documentation of a custom application. (not only the technical doc but also the manuals for end-users, including business use-cases across system frontiers if needed).

Now it's time to get an idea how we can integrate the APIs of the generated components (user-interfaces, machine-interfaces), the supported use-cases and the implemented domain classes. Since this is a rather complex issue -partially unknown to me, yet- I will try to develop this concept step-by-step in public. Any feedback is welcome.

I plan to have wxPython-based GUIs, modpython-based Web-UIs and command-line interfaces (CLI) supported by the SDK itself, but developers should be able to add other types of clients. Hence a script in Perl, a web-forum in PHP, a plug-in in VB, and so on. I guess, the CLI client will be the easiest type to start with:

Following you see a simple example of a command-line Python script foo.py. This script is implemented rather similarly to the command-line scripts of the pyswarm SDK 0.7.1. In short, it has a ScriptHandler class working as a kind of dialog-controller that is responsible for printing messages to stdout and parsing and verifying all arguments and options passed with the script at execution. Depending on the results an appropriate CommandHandler class is instantiated which is actually processing the command. (For now, let us ignore nice gimmicks like all the logging and setup features.)

OK, the ScriptHandler gets the argument bar and its value when called:

> foo.py --bar="Hello, world!"

Alternatively, bar may be just an option instead of a required parameter. So if the end-user has not provided bar, the ScriptHandler would just prompt the user to enter the string:

> foo.py
Error - No bar string provided.
Please enter bar: Hello, world!


For both script executions the result would be, that the CommandHandler would return the same string as it has been provided by the end-user:

You said: Hello, world!

So far it is not challenging. Though in a real project it would be insane to extract such implementation of the CommandHandler into a separate layer. We will do exactly this for a show-case, since I intent to work with more complex use-cases in later articles:

The use-case for this example.

As you see, we have the actor FooUser who is assigned to the use-case uses bar. Current planning suggests that this would extend to the use-case description "FooUser uses bar" in the documentatio, that would be formatted in DocBook/XML and transformable to XHTML, PDF, etc.

The documentation for the use-case would include some instructions with steps and sub-steps, eventually also with constraints, if specific conditions need to be met. Even screen-shots of the user-interface could be part of the description. However, the source for the description would be some activity (diagram) specified as implementation of this use-case. Moreover, one step (CallOperationAction) in this activity diagram would be the actual execution of a method
bar(text:String):String
as provided by the domain component. For now it is sufficient to know, that this method will return exactly the string we provided with the bar parameter.

The following illustration shows the implementation diagram for the given example. The script foo.py requires myAPI which is provided by myComponent. The APIs bar() method is implemented in the component's method with the same name.

The CLI example as implementation diagram

For outside usage of an API it is not relevant how this method is implemented inside the component. In a simple case the component itself may do the work itself, or it will pass the call to the realizing method of a particular object. However, implementations of methods can be specified in activity diagrams and the pyswarm SDK 0.7.1 already supports fully automatic generation of convenient methods (create, update, or delete object), as well as incorporation of manually implemented source code that is taboo for the generator. Alternatively, in future pyswarm releases a model-based code-generation could utilize a given activity diagram to create the implementation in a method.

Let me try with a simplified activity diagram, although I'm not sure if I use it correctly at this place:
Activity diagram for the use-case "FooUser uses bar" (click to enlarge)

Execution of the foo.py script initiates the use-case which is specified by this activity. On left hand of the foo.py swimlane you see the console content for the corresponding action. Communication between foo.py and myAPI will very likely be realized with XMLRPC, but let's leave this layer for now.

Actually, I have several issues with this "pattern".
  1. It is rather difficult to specify the same use-case with different types of clients in use.
  2. I would love if the UI client could be updated by end-users, with-out root privilege. Not just only because of the extra work for each client computer, but much more to have up-to-date clients that are definitely always in compliance with the server-side.
  3. Another thing that confused me for too long: for use-cases there are more requirements than for simple operations. Of course, use-cases and operations may have conditional flows (if...then...) and both may get rather complex. But how will the use-case continue if it requires further action by the end-user or if an error in the domain-logic occurs? There are many more turn-outs possible than for a simple API call, like myAPI.import(data):void

For sure, there are many other issues. In my next post I want to describe, how pyswarm could deal with them. If you have any thoughts on this, please feel free to leave a comment.