Hudson and the xml.meta API

at 2010-12-30 in Examples by friebe (0 comments)

Hudson Logo I'm currently working on a Hudson remoting API to be able to remotely configure our continuous integration projects. An example usecase occurred today: A co-worker had created a Hudson plugin which helps us distinguish projects with no tests at all from projects with tests exist: It display a different icon in the "weather" column and sets the build status to unstable. Now what we want to do is to enable this plugins in all projects, which would mean clicking through an odyssee of "Select Project - Configure - Enable - Save" more than 70 times in the frontend.

Hudson contains a Remote API which should be able to tackle this job.

A mix of RPC styles
Most of Hudson's remote API consist of HTTP GET requests to /api/xml - especially for discovering what you can see inside the public frontend. It gets kind of hairy for configuring Hudson projects, this is done by posting data to certain more-or-less document places:

Creating a project means posting a config.xml file as raw post data to /createItem?name=[NAME]
Updating a project's configuration works almost the same, but the target URL is /job/[NAME]/config.xml
Similarily, retrieving this config.xml can be done by issuing a GET request to the same URL
To delete a project means to perform a POST request to /job/[NAME]/doDelete?json={}&Submit=Yes

Especially the last one sounds - let's say it this way - fragile

config.xml
Anyways, the key to job configuration is the config.xml file (there seems to be no official documentation of its contents, although binging for it yields a couple of examples). Because we know how to retrieve it and how to upload it again to modify a job's configuration, and because we know it's an XML file, we would simply need to load the HTTP response's input stream it into an xml.Tree object, search this tree whether the plugin is enabled by using a custom search function, insert the correct node and post it back in order to activate the plugin. For example, to add the disk usage plugin, we would have to search whether it exists inside the properties child of the project root node.

Inside a class, we use two methods to achieve the searching part:

  class Replacer extends Object {
    // ... shortened for brevity ...
    
    protected function findProperty($properties, $pluginName) {
      foreach ($properties as $i => $property) {
        if ($pluginName === $property->getName()) return $i;
      }
      return -1;
    }
  
    public function replaceProperty(Node $pluginNode) {
      foreach ($this->tree->root->children as $child) {
        if ('properties' !== $child->getName()) continue;
        
        // Located the properties node, now replace property if existant, add otherwise
        if (-1 === ($offset= $this->findProperty($child->children, $pluginNode->getName()))) {
          $child->addNode($pluginNode);
        } else {
          $child->children[$offset]= $pluginNode;
        }
        return;
      }
    }
  }

...and then call it as follows:

  $replacer->replaceProperty(new Node('hudson.plugins.disk__usage.DiskUsageProperty'));

Much nicer would be an object-oriented API to the config.xml contents, though

Understanding the XML
To parse the XML into objects, the xml.meta.Unmarshaller class comes into place. Its unmarshalFrom() method must be passed an xml.parser.InputSource instance and a class name of which an instance is to be created.

  $unmarshaller= new Unmarshaller();
  $config= $unmarshaller->unmarshalFrom(
    new TreeInputSource($tree),
    'org.hudson_ci.core.HudsonJobConfigurationFactory'
  );

Here, two new features (available in the just released 5.7.13RC1 and in SVN head) can be seen:

The xml.parser.TreeInputSource class - as we have already parsed the XML into an xml.Tree earlier on.
The factory class - the root node can actually be one of "project", "maven2-moduleset", "matrix-project" or "hudson.model.ExternalJob" - we therefore want to create different HudsonJobConfiguration subclasses based on this fact and use a static factory method to achieve this

The basic skeleton for the factory class is:

  #[@xmlmapping(factory= 'forName')]
  class HudsonJobConfigurationFactory extends Object {
     public static XPClass forName($name) {
       switch ($name) {
         case 'project': return XPClass::forName('org.hudson_ci.core.ProjectJobConfiguration');
         // ... shortened for brevity ...
       }
     }
  }

The ProjectJobConfiguration class itself contains various methods again annotated with the @xmlmapping annotation. The easiest way is to parse nodes with just string content, for example the job's description:

  #[@xmlmapping(element = 'description')]
  public function setDescription($description) {
    $this->description= $description;
  }

Lots of the attributes contain boolean values, encoded as true for TRUE values, and false for FALSE. To cast these into the actual PHP bool datatype, we can use another new feature in the xml.meta API, casting:

  public function asHudsonBool($value) {
    switch ($value) {
      case 'true': return TRUE;
      case 'false': return FALSE;
      default: throw new IllegalArgumentException('Unrecognized boolean notation "'.$value.'"');
    }
  }

  #[@xmlmapping(element = 'keepDependencies', cast = 'asHudsonBool')]
  public function setKeepDependencies($keepDependencies) {
    $this->keepDependencies= $keepDependencies;
  }

Finally, most of the nodes inside triggers, builders and publishers are named after their fully qualified Java classname. To map these to our local classnames, we use a special factory method and pass the nodes' names:

  public function hudsonClass($name) {
    return strtr('org.hudson_ci.'.$name, array('__' => '_'));
  }
  
  #[@xmlmapping(element = 'builders/*', factory= 'hudsonClass', pass= ['name()'])]
  public function addBuilder(HudsonTask $builder) {
    $this->builders[]= $builder;
  }

For a builder task called hudson.tasks.Shell, this would yield a fully qualified class name called org.hudson_ci.hudson.tasks.Shell, which maps nicely to our local class structure. The Shell class implements a HudsonTask marker interface (as do others, e.g. Ant and Batch for example) so we can add a type restriction to the addBuilder() method and be sure we don't produce unexpected results.

Once we complete the more or less tedious task of creating @xmlmapping methods for every child node in the XML node structure, we have instances representing the XML structure, and can the easily call getter and mutator methods on them to accomplish the search and replace functionaility described above. To then get our XML to be posted back, we annotate the getters with @xmlfactory and use the xml.meta.Marshaller brother class to our unmarshaller:

  $marshaller= new Marshaller();
  $tree= new Tree($config->identifier());  // A method that returns "project", "maven2-moduleset", ...
  $marshaller->marshalTo($tree->root, $config);

Again, creating nodes with just string content inside is easy:

  #[@xmlfactory(element = 'description')]
  public function getDescription() {
    return $this->description;
  }

Casting the booleans back to their XML notation is accomplished almost the same as the other way:

  public function toHudsonBool($value) {
    return $value ? 'true' : 'false';
  }
  
  #[@xmlfactory(element = 'keepDependencies', cast = 'toHudsonBool')]
  public function getKeepDependencies() {
    return $this->keepDependencies;
  }

It gets a little bit tricky with the special lists Hudson uses (<builder class="vector"> with subnodes named after the Java class names). To accomplish this, we first have to create the list itself (the "builders" node including its class attribute) and the have to reverse the class name qualifying process, using the result as the node names. We use a little trick to create the builders node first:

  class HudsonVectorType extends Object {
    // ... shortened for brevity ...
    
    #[@xmlfactory(element = '@class')]
    public function hudsonClass() {
      return 'vector';
    }

    #[@xmlfactory(element = 'element')]
    public function getList() {
      return $this->list;
    }
  }

...and the getter exists in a usual form for programmatic access and a version special for the serialization process:

  public function getBuilders() {
    return $this.builders;
  }

  #[@xmlfactory(element = 'builders')]
  public function getBuildersVector() {
    return new HudsonVectorType($this.builders);
  }

...and finally, the classes (e.g., org.hudson_ci.hudson.tasks.Shell) themselves render their element name:

  #[@xmlfactory(element = 'name()')]
  public function hudsonName() {
    return 'hudson.tasks.Shell';
  }

The "element" in the HudsonVectorType class is just a "fake" entry, it will later be overwritten by the hudsonName() method.

That's it! The downside of this approach is obviously having to reimplement all the different XML structure elements inside config.xml, which differs greatly depending on the plugins installed (it would be nice if every Hudson plugin provided an XML schema from which we could generate the needed value objects).

The whole project will be available soon on xp-forge.net in the projects section, so stay tuned!

You can subscribe to the XP framework's news by using RSS syndication.

Hudson and the xml.meta API

Subscribe

Categories

Related