Tutorial: Custom Transformer

tl;dr: Jump to the tutorial-custom-transformer repository to see the results.

In this document we'll guide you through the steps necessary to implement a custom Transformer. Our transformer will use Babel to transpile ES2015/ES6 JavaScript to plain ES5 JavaScript that works in all Browsers.

Overview

Custom transformers provide an easy way to extend the build functionality of atscm. Basically, a transformer implements two behaviours: How atvise server nodes are mapped to files (when running atscm pull) and vice versa (when running atscm push).

Where to store transformers

Basically, transformers can be stored anywhere inside your atscm project. When using a non-ES5 configuration language (such as ES2015 or TypeScript, chosen when running atscm init) transformers should also be written in this language. atscm will handle the transpilation of your transformer code automatically. If you plan to write multiple custom transformers for your project, it is recommended to create your transformers in an own directory, e.g ./atscm.

Step 0: Project setup

In order to have the same starting point, create a new atscm project to follow this tutorial. Run atscm init and pick ES2015 as configuration language.

As for now the atvise library is written in old ES5 JavaScript, we'll ignore it in our project. Adjust your project configuration accordingly:

// Atviseproject.babel.js

...

export default class MyProject extends Atviseproject {
  ...

  static get ignoreNodes() {
    return super.ignoreNodes
      .concat(['ns=1;s=SYSTEM.LIBRARY.ATVISE']);
  }

}

Now we're ready to pull the project by running:

atscm pull

We'll use the default project files for testing later.

As suggested above, we'll store our custom transformer inside a new directory, ./atscm. Create the directory and an empty file called BabelTransformer.js:

mkdir atscm
touch atscm/BabelTransformer.js

By now you should have a project containing an ./Atviseproject.babel.js and an empty ./atscm/BabelTransformer.js file. Make sure the ./src directory contains at least the default Main display which should exist inside ./src/AGENT/DISPLAYS/Main.display.

Step 1: Import PartialTransformer class

As we don't want to implement things twice we'll subclass atscm's Transformer class. As our transformer shall only be used for JavaScript source files we can even use the PartialTransformer class which supports filtering source files out of the box. As both of these classes are exported from atscm's main file, importing them is pretty straightforward. Inside the BabelTransformer.js file add:

// atscm/BabelTransformer.js

import { PartialTransformer } from 'atscm';

Step 2: Create the BabelTransformer class

The next step is to create and export our Transformer class:

// atscm/BabelTransformer.js

import { PartialTransformer } from 'atscm';

export default class BabelTransformer extends PartialTransformer {}

We just created a PartialTransformer subclass that is exported as the file's default export. For more detailed information on ES2015's module system take a look at the docs.

Step 3: Use BabelTransformer

By default, atscm uses just some standard transformers. Any additional transformers must be configured to use inside the project's Atviseproject file.

First of all, we have to import our newly created BabelTransformer class:

// Atviseproject.babel.js

import { Atviseproject } from 'atscm'
import BabelTransformer from './atscm/BabelTransformer';

export default class MyProject extends Atviseproject { ... }

Now we override the Atviseproject.useTransformers getter to use our transformer:

// Atviseproject.babel.js

...

export default class MyProject extends Atviseproject {
  ...

  static get useTransformers() {
    return super.useTransformers
      .concat(new BabelTransformer());
  }

}

This statement tells atscm to use a new BabelTransformer instance in addition to the default transformers (super.useTransformers).

To verify everything worked so far run atscm config. Our new Transformer should show up in the useTransformers section:

$ atscm config
[08:38:16] Configuration at ~/custom-transformer/Atviseproject.babel.js
{ host: '10.211.55.4',
  port:
   { opc: 4840,
     http: 80 },
  useTransformers:
   [ DisplayTransformer<>,
     ScriptTransformer<>,
     BabelTransformer<> ],
  ...

Step 4: Implement PartialTransformer#shouldBeTransformed

PartialTransformer#shouldBeTransformed is responsible for filtering the files we want to transform. Returning true means the piped file will be transformed, false bypasses the file.

In out case we want to edit all JavaScript source files. Therefore we return true for all files with the extension .js. Edit BabelTransformer.js accordingly:

// atscm/BabelTransformer

...

export default class BabelTransformer extends PartialTransformer {

  shouldBeTransformed(file) {
    return file.extname === '.js';
  }

}

Step 5: Implement Transformer#transformFromFilesystem

Implementing Transformer#transformFromFilesystem is probably the most important part of this tutorial. In here we define the logic that actually creates ES5 code from ES2015 sources.

First of all, we need to install additional dependencies required. Running

npm install --save-dev babel-core babel-preset-2015

will install Babel and it's ES2015 preset. This preset ensures all ES5 compatible browsers will be able to run the resulting code.

We will also need the node.js buffer module. We don't need to install it, as it comes with every node installation.

Next, import these modules as usual:

// atscm/BabelTransformer.js

import { Buffer } from 'buffer';
import { PartialTransformer } from 'atscm';
import { transform } from 'babel-core';

...

The import order follows a pretty usual convention:

Core node.js modules (buffer in our case)
Other external modules (babel-core and atscm in our case)
Relative modules (./atscm/BabelTransformer.js inside Atviseproject.babel.js in our case)

Now we're ready to implement Transformer#transformFromFilesystem. What we're about to do is pretty simple:

We'll transpile the contents of the passed file with babels transform method
We clone the passed file and set it's contents to a Buffer containing the resulting code
We pass the resulting file to other streams

import ...

export default class BabelTransformer extends PartialTransformer {

  static shouldBeTransformed(file) { ... }

  transformFromFilesystem(file, enc, callback) {
    // Create ES5 code
    const { code } = transform(file.contents, {
      presets: ['es2015']
    });

    // Create new file with ES5 content
    const result = file.clone();
    result.contents = Buffer.from(code);

    // We're done, pass the new file to other streams
    callback(null, result);
  }

}

Wow! You just implemented your first custom transformer! Now we can write any scripts using the new ES2015 syntax.

Step 6: Test BabelTransformer

It's time to check if everything works as expected. Create a script file for the Main display containing ES2015 JavaScript:

// src/AGENT/DISPLAYS/Main.display/Main.js

// Class syntax
class Test {
  constructor(options = {}, ...otherArgs) {
    // Default values and rest params
    this.options = options;
    this.args = otherArgs.map((arg) => parseInt(arg, 10)); // Arrows and Lexical This
  }
}

const a = 13; // Constants
const { options, args } = new Test({ a }, '23'); // Enhanced Object Literals

alert(`Option a: ${options.a}, args: ${args.join(', ')}`); // Template Strings

Run atscm push to upload the new display script to atvise server. Open your atvise project in your favorite browser (you may have to delete the browser cache) and if everything worked you should see an alert box containing the text "Option a: 13, args: 23". When you inspect the page's source you'll see the display script code was transpiled to ES5.

Step 7: Implement Transformer#transformFromDB

As said at the beginning, atscm transformers allow transformation from and to the filesystem. A babel transpilation is a one-way process, meaning you cannot create ES2015 source code from the resulting ES5 code. Therefore the only thing we can do when transforming from atvise server to the filesystem is to prevent an override.

We do so by implementing Transformer#transformFromDB:

// atscm/BabelTransformer.js
...

export default class BabelTransformer extends PartialTransfromer {
  ...

  transformFromDB(file, enc, callback) {
    // Optionally, we could print a warning here
    callback(null); // Ignore file, remove it from the stream
  }
}

Now we can run atscm push without overriding our ES2015 source code.

Result

This is how your custom transformer should look now:

// atscm/BabelTransformer.js

import { Buffer } from 'buffer';
import { PartialTransformer } from 'atscm';
import { transform } from 'babel-core';

export default class BabelTransformer extends PartialTransformer {
  shouldBeTransformed(file) {
    return file.extname === '.js';
  }

  transformFromFilesystem(file, enc, callback) {
    // Create ES5 code
    const { code } = transform(file.contents, {
      presets: ['es2015'],
    });

    // Create new file with ES5 content
    const result = file.clone();
    result.contents = Buffer.from(code);

    // We're done, pass the new file to other streams
    callback(null, result);
  }

  transformFromDB(file, enc, callback) {
    callback(null); // Ignore file, remove it from the stream
  }
}

Conclusion

We just created a custom Transformer in no time. It transpiles ES2015 code on push and prevents overriding this code on pull.

Of course there are many ways to improve the transformer, for example:

Handle options to configure how babel transpiles the source code