Holistic Application Platform

Holistic means:

  • Open source - Based on free and open source components
  • Productive - From setting up the developer environment to provisioning and monitoring servers in production
  • Web-based - JSON based WebSocket and HTTP interfaces with Java and JavaScript clients
  • Customer focused - Made to create applications with customer specific deployments
  • Extendable - Meta model to represent any domain and extensible backend to plug-in domain specific services
  • Buildable - Unified build system for all artifacts + SCM + build server + software repository
  • Testable - Write and run unit, UI, integration and deployment tests
  • Deployable - JVM + single JAR + static web apps + automated provisioning on *nix and Windows servers

Platform enables smaller developers to:

  • Setup a professional development environment for their team
  • Build domain specific applications based on a solid, extensible foundation
  • Thoroughly test the developed applications in an automated way
  • Build modules for the automated installation of applications
  • Create customer specific (or otherwise differentiated) versions of modules
  • Provision hosted, on-premise or cloud servers with modules
  • Monitor application runtimes and provide great support to customers

Contact and help:

  • IRC channel #platform on OFTC (WebChat)
  • platform@rswk.ch

Content:

Guidelines

Built for:

  • Small and medium size business
  • Problems, domains and applications that are modeled or interact with physical objects
  • Microservices, Monoliths, Sphagetti, Lasagna and Ravioli architectures

Not built for:

  • The Next Web-Scale Twooglebook

Principles:

  • Automate but not automagically
  • Keep up to date
  • Minimize abstractions
  • Minimize dependencies
  • Minimize lockin
  • Monorepos
  • Move fast but try to not break too many things
  • Remove cruft and YAGNI
  • Sometimes boring is better: Capabilities and failure modes are well understood

Formatting

Setup formatters/extensions according to setup guide.

  • Java/Xtend: Ctrl+Shift+F and Ctrl+Shift+O in Eclipse
  • CSS: Shift+Alt+F in Visual Studio Code
  • HTML: Shift+Alt+F in Visual Studio Code
  • JavaScript: Manual formatting so it conforms to JSLInt
  • JSON: Shift+Alt+F in Visual Studio Code
  • XML: Shift+Alt+F in Visual Studio Code
  • Markdown: Manual formatting

Components

Documentation for all central framework components and modules in platform source repository, including sample code.

Technical basics:

  • All binary artifacts are built with Maven, or at least Maven triggers the specific build systems
    • Server side Xtend code is transpiled to Java and then packaged as JAR or Shaded JAR
    • Client side HTML, CSS and JavaScript resources are bundled using Node.js with r.js and packaged into static ZIP archives with Maven
    • Android apps are built using Gradle and published to the Maven repository
  • Java uses Dagger for dependency injection and code is structured into different Maven modules and Java packages as usual
  • JavaScript uses AMD for dependency injection and to break code reusable modules

Top level source repository structure:

- android
- bootstrap
- framework
  - bootstrap
  - ide
  - meta
  - util
  - web
- pom.xml

Meta

Ist das Herzstück und Backend jeder Applikation. Ein Verbund von zentralen Services mit Websocket und HTTP basierten Schnittstellen für Clients. Java und JavaScript Implementationen der Client-Schnittstellen für die Einbindung in Applikationen sind enthalten, wie auch eine Admin Web-Applikation.

meta-core

Sind alle gemeinsamen Klassen für das Meta-Modell und Protokoll zwischen Client und Server, jeweils in Java und JavaScript.

Definiert das Meta-Modell auf welchem die Domänen spezifischen Modelle aufgebaut werden können:

  • MetaType - Definiert einen Typ anhand eines JSON-Schemas mit beliebigen Eigenschaften, welches zur Validierung von Objekten dieses Typs verwendet wird. Beispiel: Typ “Buch” mit Eigenschaft “Titel”
{
    "meta": "type",
    "id": "bfcf07388afd45faa06e259548e39ecd",
    "name": "writeAndSearch_type",
    "schema": {
        "$schema": "http://json-schema.org/draft-04/schema#",
        "type": "object",
        "required": [
            "owner",
            "picture",
            "origin"
        ],
        "additionalProperties": false,
        "properties": {
            "owner": {
                "title": "Owner",
                "description": "The owner of this product",
                "type": "string"
            },
            "picture": {
                "title": "Picture",
                "description": "A picture of this product (base64)",
                "type": "string"
            },
            "origin": {
                "title": "Origin",
                "description": "The origin of this product (latlng)",
                "type": "object",
                "properties": {
                    "lat": {
                        "type": "number"
                    },
                    "lng": {
                        "type": "number"
                    }
                }
            },
            "new": {
                "type": "boolean"
            }
        },
        "definitions": {}
    }
}
  • MetaObject - Definiert das konkrete Objekt eines Typs und enthält die Eigenschaften in JSON. Beispiel: Objekt vom Typ “Buch” mit Titel “The Lord of the Rings”
{
    "meta": "object",
    "id": null,
    "name": "writeAndSearch_object",
    "typeId": "bfcf07388afd45faa06e259548e39ecd",
    "value": {
        "owner": "My Company Inc.",
        "picture": "...",
        "origin": {
            "lat": 47.07386310181414,
            "lng": 7.899169921874999
        },
        "new": false
    },
    "position": null
}
  • MetaAction - Definiert applikationsspezifische Aktionen die nicht über die öffentlichen Schnittstellen abgehandelt werden können in Form von JSON-Schemas für die Parameter und Resultat der Aktion. Beispiel: Aktion “Buch ausleihen” mit Parameter “Exemplarnr.” und Resultat “Fälligkeitsdatum”
{
    "meta": "action",
    "id": "...",
    "name": "Test Action",
    "paramSchema": {
        "$schema": "http://json-schema.org/draft-04/schema#",
        "type": "object",
        "required": [...],
        "additionalProperties": false,
        "properties": {...},
        "definitions": {}
    },
    "resultSchema": {
        "$schema": "http://json-schema.org/draft-04/schema#",
        "type": "object",
        "required": [...],
        "additionalProperties": false,
        "properties": {...},
        "definitions": {}
    }
}
  • MetaLogEntry - Repräsentiert Veränderungen an Typen, Objekten und Aktionen über Zeit. Werden für CUD Operationen automatisch vom System erfasst, können aber für spezielle Ereignisse auch manuell erzeugt werden
{
    "meta": "log",
    "id": "...",
    "name": null,
    "metaType": "object",
    "metaId": "456",
    "metaName": "MetaServiceTest.write_update_delete",
    "timestamp": "2019-10-16T11:51:48.314Z",
    "type": "UPDATE",
    "message": "write",
    "data": {
        "value": {
            "owner": "Another Company Inc."
        },
        "position": {
            "lat": 3.0,
            "lng": 4.0
        },
        "change": [
            "owner",
            "position"
        ],
        "typeId": "123"
    }
}

Somit ergibt sich ein JSON-Schema und JSON basiertes Modell von Typen und Objekten, deren Veränderungen über Zeit festgehalten werden. Zusammen mit den eingebauten Geospatial-Eigenschaften ist das Modell ideal zur Abbildung und Anbindung von physischen Objekten im Kontext von Auto-ID/IoT. Mit den im Client-Protokoll definierten ClientNotification und MetaNotification können Applikationen gebaut werden, welche in Echtzeit Ihre Änderungen mit allen angeschlossenen Clients teilen.

Zum Modell gibt es auch Benutzer, Gruppen und Konfigurationen. Berechtigungen auf Benutzer/Gruppen erlauben es den Zugriff auf Schnittstellen, Typen, Objekte, Aktionen, etc. einzuschränken.

  • MetaUser
{
    "name": "pw",
    "password": "pw",
    "phone": null,
    "email": null,
    "webhook": null,
    "totp": null,
    "grants": { // See MetaGrantKeys / MetaGrantPermissions
        "ac": 1,
        "cl": 1,
        "cln": 1,
        "dl": 1,
        "fs": 1,
        "li": 1,
        "men": 1,
        "nc": 1,
        "se": 1,
        "sy": 1,
        "tk": 1,
        "up": 1,
        "wr": 1,
        "wsst": 15
    }
}
  • MetaUserGroup - Einer Gruppe werden Benutzer und Berechtigungen zugeteilt. Berechtigungen werden zuerst auf dem Benutzer geprüft und dann auf seinen zugehörigen Gruppen. Die Gruppe admin wird speziell gehandhabt (kann z.B. nicht gelöscht werden)
{
    "name": "admin",
    "users": [
        "pw"
    ],
    "grants": {
        "test": 3,
        "egn": 3
    }
}
  • MetaUserConfig - Konfigurationen für den aktuellen Benutzer und seine Gruppen werden automatisch in der LoginReply mitgeliefert. Eine Konfiguration kann einem Benutzer, einer Gruppe oder einer Kombination von beidem zugeordnet werden.
{
    "user": null,
    "group": "admin",
    "config": {
        "isBest": true
    }
}

meta-client

Client implementations in Java and JavaScript of all APIs provided by the meta-engine server.

Java Client API
def ListenableFuture<Void> connect() 

def void disconnect()

def void destroy()

def boolean isConnected()

def boolean isReconnecting()

def boolean isAuth()

def ListenableFuture<LoginReply> login(LoginRequest it, RequestOption... options)

def ListenableFuture<AuthReply> auth(AuthRequest it, RequestOption... options)

def ListenableFuture<TokenReply> token(TokenRequest it, RequestOption... options)

def ListenableFuture<SubscriptionReply> subscribe(SubscriptionRequest it, RequestOption... options)

def ListenableFuture<LogoutReply> logout(LogoutRequest it, RequestOption... options)

def void postClientNotification(ClientNotification it) {

def ListenableFuture<ClientListReply> getClientList(ClientListRequest it, RequestOption... options)

def ListenableFuture<NotificationCacheReply> getNotificationCache(NotificationCacheRequest it, RequestOption... options)

def ListenableFuture<WriteReply> write(WriteRequest it, RequestOption... options)

def ListenableFuture<SearchReply> search(SearchRequest it, RequestOption... options)

def ListenableFuture<ActionReply> execute(ActionRequest it, RequestOption... options)

def void log(LoggingMessage it)

def ListenableFuture<LoggingEntrySearchReply> loggingEntrySearch(LoggingEntrySearchRequest it, RequestOption... options)

def ListenableFuture<LoggingMetricSearchReply> loggingMetricSearch(LoggingMetricSearchRequest it, RequestOption... options)

def ListenableFuture<LoggingDeleteReply> loggingDelete(LoggingDeleteRequest it, RequestOption... options)

def ListenableFuture<UserWriteReply> userWrite(UserWriteRequest it, RequestOption... options)

def ListenableFuture<UserSearchReply> userSearch(UserSearchRequest it, RequestOption... options)

def ListenableFuture<UserGroupWriteReply> groupWrite(UserGroupWriteRequest it, RequestOption... options)

def ListenableFuture<UserGroupSearchReply> groupSearch(UserGroupSearchRequest it, RequestOption... options)

def ListenableFuture<UserConfigWriteReply> configWrite(UserConfigWriteRequest it, RequestOption... options)

def ListenableFuture<UserConfigSearchReply> configSearch(UserConfigSearchRequest it, RequestOption... options)

def ListenableFuture<UploadReply> upload(UploadRequest it, RequestOption... options)

def ListenableFuture<UploadReply> uploadAndWaitDone(UploadRequest it, Path file)

def ListenableFuture<DownloadReply> download(DownloadRequest it, RequestOption... options)

def ListenableFuture<DownloadReply> downloadAndWaitDone(DownloadRequest it, Path file)

def ListenableFuture<FileDeleteReply> deleteFile(FileDeleteRequest it, RequestOption... options)

def ListenableFuture<FileSearchReply> fileSearch(FileSearchRequest it, RequestOption... options)

def ListenableFuture<BackupReply> backup(BackupRequest it, RequestOption... options)

def ListenableFuture<BackupRestoreReply> backupRestore(BackupRestoreRequest it, RequestOption... options)

def ListenableFuture<WatchdogUpdateReply> processLocalUpdate(WatchdogUpdateRequest it, RequestOption... options)

def ClientSubscription registerOnEvent(Procedure1<Event> it)

def void unregister(ClientSubscription it)

def void registerDownloadListener(DownloadListener downloadListener)
JavaScript Client API
that.connect = function () {};

that.disconnect = function () {};

that.destroy = function () {}; // Not implemented

that.isConnected = ko.observable();

that.isReconnecting = ko.observable();

that.isAuth = ko.observable();

that.login = function (request, options) {};

that.auth = function (request, options) {};

that.token = function (request, options) {};

that.subscribe = function (request, options) {};

that.logout = function (request, options) {};

that.postClientNotification = function (clientNotification) {};

that.getClientList = function (request, options) {};

that.getNotificationCache = function (request, options) {};

that.write = function (request, options) {};

that.search = function (request, options) {};

that.execute = function (request, options) {};

that.log = function (loggingMessage) {};

that.loggingEntrySearch = function (request, options) {};

that.loggingMetricSearch = function (request, options) {};

that.loggingDelete = function (request, options) {};

that.userWrite = function (request, options) {};

that.userSearch = function (request, options) {};

that.groupWrite = function (request, options) {};

that.groupSearch = function (request, options) {};

that.configWrite = function (request, options) {};

that.configSearch = function (request, options) {};

that.upload = function (request, options) {};

that.uploadAndWaitDone = function (request, buffer) {};

that.download = function (request, options) {};

that.downloadAndWaitDone = function (request) {};

that.fileDelete = function (request, options) {};

that.fileSearch = function (request, options) {};

that.backup = function (request, options) {};

that.backupRestore = function (request, options) {};

that.processLocalUpdate = function (request, options) {};

that.registerOnEvent = function (subscription) {};

that.unregister = function (clientSubscription) {};

that.registerDownloadListener = function (listener) {}; // Not implemented

meta-engine

Is the backend/server runtime of any application based on meta.


Various Services contain the the vital functions. Each service is one of:

  • AbstractEngineIdleService - Similar to AbstractIdleService
  • AbstractEngineExecutionThreadService - Similar to AbstractExecutionThreadService
  • AbstractEngineWorkQueueService - Custom version of AbstractEngineExecutionThreadService with a work queue
ActionService.xtend

Receives and dispatches MetaAction to services that can handle them. Does validation of parameters and results, in case schemas are defined.

Interface: None

Configuration: None

Usage:

  1. Define a MetaAction in Java and JavaScript
// Model.xtend

val public static ECHO_ACTION_ID = "echo"
val public static META_ECHO_ACTION = new MetaAction(ECHO_ACTION_ID, "Echo Action", EMPTY_OBJECT_SCHEMA, EMPTY_OBJECT_SCHEMA)

@Jsonized @Data class SimpleServiceEchoActionParam {
    String ping
}

@Jsonized @Data class SimpleServiceEchoActionResult {
    String pong
}
// model.js

that.ECHO_ACTION_ID = "echo";

factory.simpleServiceEchoActionParam = function (spec) {
    const param = {};

    param.ping = spec.ping;

    return param;
};

factory.simpleServiceEchoActionResult = function (spec) {
    const result = {};

    result.pong = spec.pong;

    return result;
};
// protocol.js

factory.simpleServiceEchoActionRequest = function (params) {
    return metaProtocol.factory.actionRequest({
        actionId: model.ECHO_ACTION_ID,
        params: model.factory.simpleServiceEchoActionParam(params)
    });
};

factory.simpleServiceEchoActionReply = function (result) {
    return metaProtocol.factory.actionReply({
        result: model.factory.simpleServiceEchoActionResult(result)
    });
};
  1. Service to handle the actions
@Singleton class SimpleService extends AbstractEngineIdleService<SimpleServiceConfig> {
    @Inject new(EngineRuntime runtime, NotificationService notificationService, SimpleServiceConfig config) {
        super(runtime, notificationService, config)
    }

    override protected startUp() {
        if(runtime.awaitDependentServices(notificationService) === false) {
            return
        }

        LOG.info("startUp, config={}", config)
        super.startUp()
    }

    @Subscribe def void onActionExecutionRequestNotification(ActionExecutionRequestNotification it) {
        switch (action.id) {
            case ECHO_ACTION_ID: {
                runActionExecutionRequest(runtime, notificationService, it, SimpleServiceEchoActionParam, [echo], SYSTEM_ERROR)
            }
        }
    }

    def ListenableFuture<SimpleServiceEchoActionResult> echo(SimpleServiceEchoActionParam it) {
        checkRunning
        LOG.info("echo")
        return Futures.immediateFuture(new SimpleServiceEchoActionResult(ping))
    }
}
  1. Execute action from another service
TODO
  1. Execute action from Java client
val params = new SimpleServiceEchoActionParam("Hello")
val reply = client.execute(ActionRequest.forgeFromObject(ECHO_ACTION_ID, params)).get(2, TimeUnit.SECONDS)
val result = reply.meltResult(SimpleServiceEchoActionResult)
  1. Execute action from JavaScript client
return client.execute(protocol.factory.simpleServiceEchoActionRequest({
    ping: "Hello"
})).then(function (actionReply) {
    LOG.debug("pong=" + actionReply.result.pong);
});
DatabaseService.xtend

Allows to connect to any configured SQL databases using c3p0. Executing queries against the databases is handled by JDBI.

Interface:

def Jdbi lookupDefaultDatabase()

def Jdbi lookupDatabase(String dbName)

Configuration:

"database": {
    "dbs": [
        {
            // This would be the default database, no name property given
            "driver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
            "jdbcUrl": "jdbc:sqlserver://...",
            "user": "admin",
            "password": "UeDgud7x"
        },
        {
            "name": "mariadb",
            "driver": "org.mariadb.jdbc.Driver",
            "jdbcUrl": "jdbc:mariadb://..."
        }
    ]
}

Make sure to include the required driver JARs in the Maven POM or classpath at runtime.

Usage:

  1. From another service
@Singleton class SimpleService extends AbstractEngineIdleService<SimpleServiceConfig> {
    DatabaseService databaseService

    @Inject new(EngineRuntime runtime, DatabaseService databaseService, SimpleServiceConfig config) {
        super(runtime, null, config)
        this.databaseService = databaseService
    }

    override protected startUp() {
        if(runtime.awaitDependentServices(databaseService) === false) {
            return
        }

        LOG.info("startUp, config={}", config)
        super.startUp()
    }

    def void sayHelloWithDao(String name) {
        val dao = databaseService.lookupDefaultDatabase.onDemand(SimpleServiceHelloDao)
        dao.insertName(name)
    }

    def void sayHelloWithHandler(String name) {
        databaseService.lookupDatabase("mariadb").withHandle [
            execute(SimpleServiceHelloDao.CREATE_TABLE_QUERY)
            execute(SimpleServiceHelloDao.INSERT_QUERY, name)
        ]
    }
}

interface SimpleServiceHelloDao {
    val static CREATE_TABLE_QUERY = '''CREATE TABLE Hello (
        Name VARCHAR(20) NOT NULL
    )'''
    val static INSERT_QUERY = "INSERT INTO Hello (Name) VALUES (?)"

    @SqlUpdate(CREATE_TABLE_QUERY)
    def void createHelloTable()

    @SqlUpdate(INSERT_QUERY)
    def void insertName(String name)
}
FileService.xtend

Upload, download, search and delete files. Provides backup and restore functionality to meta-engine services. Uses a LuceneService instance to store file meta data. File contents are stored on disk.

Interface:

def ListenableFuture<Pair<FileInfo, Path>> localDownload(DownloadRequest it, @Nullable Path existingTarget)

def ListenableFuture<UploadReply> localUpload(UploadRequest it, Path file)

def ListenableFuture<FileDeleteReply> fileDelete(FileDeleteRequest it)

def ListenableFuture<FileSearchReply> fileSearch(FileSearchRequest it)

def ListenableFuture<BackupReply> backup(BackupRequest it)

def ListenableFuture<BackupRestoreReply> backupRestore(BackupRestoreRequest it)

interface FileServiceBackupClient {
    def void onFileServiceBackupNotification(FileServiceBackupNotification it)

    def void onFileServiceRestoreNotification(FileServiceRestoreNotification it)
}

Configuration:

TODO

Usage:

  1. From another service
val uploadFuture = fileService.localUpload(uploadRequest, path)
val downloadFuture = fileService.localDownload(downloadRequest, null)
Futures.transform(downloadFuture, [ fileInfoAndPath |
    val info = fileInfoAndPath.key
    val path = fileInfoAndPath.value
    ...
    deleteIfExists(path)
], runtime)
  1. From Java client
val uploadDoneFuture = client.uploadAndWaitDone(uploadRequet, path)
val downloadDoneFuture = client.downloadAndWaitDone(downloadRequest, path)
  1. Upload in JavaScript client
// html
<input type="file" class="custom-file-input" data-bind="event: {change: loadCsvFile}">
that.loadCsvFile = function (ignore, event) {
    LOG.debug("loadCsvFile");
    return ext.loadFileContent(event, "buffer", null, [".csv"]).then(function (loadedFile) {
        return client.uploadAndWaitDone(protocol.factory.uploadRequest({
            name: loadedFile.name,
            tags: ["csv"]
        }), loadedFile.content);
    }).then(function (uploadReply) {
        LOG.debug("fileKey=" + uploadReply.key);
    });
};
  1. Download in JavaScript client
return client.downloadAndWaitDone(protocol.factory.downloadRequest({
    key: fileKey
})).then(function (downloadReply) {
    LOG.debug("fileInfo=" + downloadReply.file);
    // Do something with downloadReply.buffer
});
GatewayService.xtend

Sends SMS, Emails and is able to invoke webhooks.

SMS delivery depends on an external provider: iNetWorx SMS-Gateway.

Interface:

def ListenableFuture<GatewayServiceDeliveryReply> sendSms(GatewayServiceSmsRequest it)

def ListenableFuture<GatewayServiceDeliveryReply> sendEmail(GatewayServiceEmailRequest it)

def ListenableFuture<GatewayServiceDeliveryReply> invokeWebhook(GatewayServiceWebhookRequest it)

Configuration:

TODO

Usage:

  1. From another serivce
val file = new GatewayServiceEmailFileRequest(true, "/9j/4AAQSkZJRgABAQAA...", "image/jpg", "picture.jpg", null)
val emailFuture = gatewayService.sendEmail(new GatewayServiceEmailRequest("garbage@thedumpster.com", "Hello World", null, #[file]))
LoggingService.xtend

Centralized store and search for logs and metrics. Uses a LuceneService instance to store data.

Internally attatches to SLF4J root logger to log everything from the server runtime. Clients are able to log to this, as well as use server side metrics.

All messages that contain parameters in the format arg1=value, arg2=value, ... are automatically indexed and can be searched. All field names (arg1) are sanitized, see sanitizeStringFieldValue.

Interface:

def ListenableFuture<LoggingEntrySearchReply> loggingEntrySearch(LoggingEntrySearchRequest it)

def ListenableFuture<LoggingMetricSearchReply> loggingMetricSearch(LoggingMetricSearchRequest it)

def ListenableFuture<LoggingDeleteReply> loggingDelete(LoggingDeleteRequest it)

Configuration:

TODO

Usage:

  1. From another service
import static com.codahale.metrics.MetricRegistry.name

@Singleton class SimpleService extends AbstractEngineIdleService<SimpleServiceConfig> {
    val static LOG = LoggerFactory.getLogger(SimpleService)

    Meter helloMeter

    @Inject new(EngineRuntime runtime, SimpleServiceConfig config) {
        super(runtime, null, config)
    }

    override protected startUp() {
        LOG.info("startUp, config={}", config)
        helloMeter = runtime.metricRegistry.meter(name(this.class, "hello"))
        super.startUp()
    }

    def void sayHello() {
        LOG.debug("sayHello")
        helloMeter.mark
    }
}
  1. Logging and metrics from Java client
class ApplicationClient {
    val static LOG = LoggerFactory.getLogger(ApplicationClient)

    def static void main(String... args) {
        val clientConfig = new ClientConfig("app.mycompany.ch", null, 0, 0, 0, 0)
        val client = new Client(clientConfig)

        LOG.info("connecting client")
        client.connect.get

        val appender = Logging.installEngineAppender(LOG, client) // Everything from this point is also logged on the server
        val loginTimer = Logging.forgeMetricTimer(client, "ApplicationClient-login")
        val timer = loginTimer.time

        val loginRequest = new LoginRequest("pw", "pw", "ApplicationClient", Cst.VERSION)
        val loginReply = client.login(loginRequest).get
        val authRequest = new AuthRequest(loginReply.otp)
        client.auth(authRequest).get
        timer.stop

        LOG.info("shutting down")
        client.logout(new LogoutRequest).get
        appender.stop
        client.destroy
    }
}
  1. Logging and metrics from JavaScript client
define([
    "logger", "myapp-web/config", "meta-client/client", "meta-client/logging", "meta-core/protocol", "domReady!"
], function (Logger, config, metaClient, logging, protocol) {
    "use strict";

    const LOG = Logger.get("myapp-web/index");
    const that = Object.create(null);

    that.client = metaClient({
        workerPath: config.workerPath,
        engine: config.engine
    });
    logging.installEngineLogger(that.client, (
        config.debug
        ? Logger.DEBUG
        : Logger.INFO
    )); // Everything from this point is also logged on the server

    that.loginTimer = logging.forgeMetricTimer(that.client, "myapp-web/login");

    that.login = function () {
        const timer = that.loginTimer.time();
        return that.client.login(protocol.factory.loginRequest({
            user: "pw",
            password: "pw",
            clientName: "myapp-web",
            clientVersion: config.version
        })).then(function (loginReply) {
            return that.client.auth(protocol.factory.authRequest({
                otp: loginReply.otp
            }));
        }).then(function () {
            timer.stop();
        });
    };

    that.client.isConnected.subscribe(function (isConnected) {
        if (isConnected) {
            that.login().then(function () {
                LOG.info("connected and loggedin");
            });
        }
    });

    that.client.connect();
});
  1. Search from JavaScript client
const query = `clientId:${client.clientId()}`;
const from = metaCst.now().subtract(1, "hour");
const to = metaCst.now();
const timestampRangeQuery = ` AND timestamp:{${metaCst.toDateTimeNumber(from)} TO ${metaCst.toDateTimeNumber(to)}}`;
const searchRequest = protocol.factory.loggingEntrySearchRequest({
    query: query + dateRangeQuery,
    limit: 1000
});
client.loggingEntrySearch(searchRequest).then(function (searchReply) {
    console.log(searchReply);
});
LuceneService.xtend

Wrapper for working with a Lucene index to persist JSON entities. Mapping JSON entities to Lucene documents happens in DocumentForge.

For some entities like MetaObject and MetaLogEntry the generated document contains dynamic fields generated from the specific contents. The document for a MetaObject contains (among others) the following fields that are always present:

  • id - The objects UUID
  • typeId - The UUID from the MetaType that this object is an instance of

For a MetaObject with the following value:

{
    "owner": "My Company Inc.",
    "flagged": true
}

DocumentForge will generate additional fields with createLuceneFields: owner and flagged. With the JSON schema of the MetaType we:

  • Define what fields are sortable
  • What fields are indexed as numeric values
{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "required": [],
    "sortable": ["myNumber"],
    "additionalProperties": false,
    "properties": {
        "myNumber": {
            "title": "Number Sortable",
            "description": "This field is going to be sortable, since we added it to the 'sortable' list of fields.",
            "type": "number"
        }
    },
    "definitions": {}
}

The fields in a document can be further customized by using the AbstractDocumentForgeFieldCustomizer to:

  • Generate additional fields
  • Remove fields that would be automatically generated
  • Register additional fields as numeric

Customizer to add an additional numeric field:

class ArraySizeField extends AbstractDocumentForgeFieldCustomizer {
    val static SIZE_FIELD = "arraySize"

    override forgeFields(String meta, String name, JsonNode node, boolean isSortable) {
        if(META_TYPE_OBJECT == meta && node.array) {
            return #{DocumentForge.forgeNumberField(SIZE_FIELD, root.size)} -> false
        }
    }

    override forgePointFieldNames(String name, JsonNode node, String typeId) {
        return #{SIZE_FIELD}
    }
}

Customizer to ignore certain fields:

class IgnoreObjectFields extends AbstractDocumentForgeFieldCustomizer {
    val static IGNORED_FIELDS = #{"field1", "field2"}

    override forgeFields(String meta, String name, JsonNode node, boolean isSortable) {
        if(IGNORED_FIELDS.contains(name)) {
            return #{} -> true
        }
    }
}

Multiple instances of LuceneService are used for FileService, MetaService, LoggingService and UserService. Services that need their own Lucene index must create their own LuceneService instance. Never interface with another LuceneService directly, use the interface given by its owner. See the mentioned meta-engine services as reference.

Interface:

def <T> ListenableFuture<T> search(Function1<LuceneServiceSearchContext, T> search)

def <T> ListenableFuture<T> write(Function1<LuceneServiceWriteContext, T> write)

def <T> ListenableFuture<T> searchAndWrite(Function1<LuceneServiceSearchAndWriteContext, T> searchAndWrite

public ConcurrentHashMap<String, String> getExtendedCommitData()

Configuration:

TODO

Usage:

TODO
MetaService.xtend

CRUD service for all meta entities (MetaLogEntry, MetaType, MetaObject, MetaAction). Uses a LuceneService instance to store them.

Interface:

def ListenableFuture<SearchReply> search(SearchRequest it)

def ListenableFuture<WriteReply> write(WriteRequest it)

Configuration: None

Usage:

  1. From another service
val request = SearchRequest.forge(META_OBJECT_QUERY, 0)
return Futures.transform(metaService.search(request), [ searchReply |
    val numberOfObjects = searchReply.metas.size;
    LOG.debug("numberOfObjects={}", numberOfObjects)
    return numberOfObjects
], runtime)
  1. From Java cient
val request = SearchRequest.forge(META_OBJECT_QUERY, 0)
val reply = client.search(request).get
  1. From JavaScript client
return client.search(protocol.factory.searchRequest({
    query: META_OBJECT_QUERY,
    limit: 0,
    sortField: "timestamp",
    sortReverse: true
})).then(function (searchReply) {
    var numberOfObjects = searchReply.metas.length;
    LOG.debug(`numberOfObjects=${numberOfObjects}`);
    return numberOfObjects;
});
NotificationService.xtend

Wrapper for EventBus to publish and subscribe to notifications. Keeps a local cache to allow clients that were offline to catch up.

Watches the server JSON config files and notifies services via AbstractNotificationServiceConfigWatcher if anything changes. See EngineConfigWatcher.

Interface:

def void register(Object it)

def void unregister(Object it)

def void post(Object it)

def ListenableFuture<NotificationCacheReply> getNotificationCache(NotificationCacheRequest it)

Configuration:

TODO

Usage:

  1. From another service
import com.google.common.eventbus.Subscribe

@Singleton class SimpleService extends AbstractEngineIdleService<SimpleServiceConfig> {
    @Inject new(EngineRuntime runtime, NotificationService notificationService, SimpleServiceConfig config) {
        super(runtime, notificationService, config) // Will register this service with notificationService
    }

    override protected startUp() {
        if(runtime.awaitDependentServices(notificationService) === false) {
            return
        }

        LOG.info("startUp, config={}", config)
        super.startUp()
    }

    @Subscribe def void onSimpleServiceNotification(SimpleServiceNotification it) {
        // Invoked when ever someone posts a SimpleServiceNotification
    }

    def void postNotification() {
        notificationService.post(new SimpleServiceNotification("Hello"))
    }
}

@Data class SimpleServiceNotification {
    String greeting
}
  1. From Java client
client.registerOnEvent[
    switch (it) {
        ClientNotification: {
            // We received a ClientNotification
        }
    }
]
client.postClientNotification(ClientNotification.newNow(null, null, "mychannel", "INFO", "Hello"))
  1. From JavaScript client
client.registerOnEvent(function () {
    var subscription = Object.create(null);
    subscription[protocol.CLIENT_NOTIFICATION] = function (clientNotification) {
        // We received a ClientNotification
    };
    return subscription;
}());
client.postClientNotification(protocol.factory.clientNotification({
    channel: "mychannel",
    action: "INFO",
    message: "Hello"
}));
SyncService.xtend

Experimental service to synchronize files, meta entities, notifications and users across a cluster of meta-engine runtimes. etcd is used as the distributed key-value store and at least one node is required to synchronize multiple meta-engine runtimes.

Services are registered to watch for changes on the last locally commited revision of their key. Since etcd does not store revisions for a key indefinitely, services must be able to catch up to the quorum by replicating the complete state from another node by means outside of etcd. After replication, it can try to watch the key from the revision that was last commited.

Services must register their delegates with SyncService on startup. The delegate will either route the request through the etcd cluster or do a simple local invocation, depending on the configuration at runtime.

Interface:

def void registerServiceDelegate(AbstractSyncServiceDelegate serviceDelegate)

public io.etcd.jetcd.Client getEtcdClient()

interface SyncServiceDelegate<T, R> {
    def long getLatestRevision()

    def ListenableFuture<R> write(T request, @Nullable Map<String, String> context)

    def ListenableFuture<R> onNext(T request, @Nullable Map<String, String> context, long revision)
}

Configuration:

TODO

Usage:

  1. From another service
class MyService {
    override protected startUp() {
        (...)
        serviceDelegate = new MyServiceSyncDelegate(this)
        syncService.registerServiceDelegate(serviceDelegate)
        (...)
    }

    def ListenableFuture<MyServiceResponse> doIt(MyServiceRequest request) {
        return serviceDelegate.write(it, syncContext)
    }

    def protected ListenableFuture<MyServiceResponse> processDoIt(MyServiceRequest it, @Nullable Map<String, String> syncContext, long revision) {
        return Futures.immediateFuture(new MyServiceResponse)
    }
}

class MyServiceSyncDelegate extends AbstractSyncServiceDelegate<MyServiceRequest, MyServiceResponse> {
    val static KEY = "myservice"
    val static REQUEST_READER = OM.readerFor(MyServiceRequest)
    val static REQUEST_WRITER = OM.writerFor(MyServiceResponse)

    MyService myService

    new(MyService myService) {
        super(KEY, REQUEST_READER, REQUEST_WRITER)
        this.myService = myService
    }

    override getLatestRevision() {
        return myService.latestCommitedRevision
    }

    override onNext(WriteRequest request, @Nullable Map<String, String> context, long revision) {
        return myService.processDoIt(request, context, revision)
    }
}
UserService.xtend

CRUD service for all user entities (MetaUser, MetaUserGroup, MetaUserConfig). Provides login, authentication, authorization, verification and other access control functionality. Uses a LuceneService to store them.

Field name of MetaUser is sanitized to letters and digits. Valid names can be pw, Hello123, but not super_user.

Passwords are stored using SCrypt (wg/scrypt). Passwords are optional since it’s possible to use OTP. OTPs are 6 digit numbers like 123 456. They are sanitized before validation, so ASD123ASD456, 1 2 3!456 would be valid OTPs. OTPs can be SMS, Email, webhook or TOTP.

Verification of a MetaUser can be done with SMS, Email or a webhook. This can be used to temporarily grant a user certain permissions, if he can verify that he owns the specific phone number, email or URL.

Interface:

def ListenableFuture<TokenReply> token(TokenRequest it, String clientId)

def Pair<Boolean, Jws<Claims>> verifyToken(String token, @Nullable String user, @Nullable Map<String, Integer> requiredGrants)

def ListenableFuture<String> verifyUser(MetaUser it, String clientId)

def ListenableFuture<String> verifyUserOtp(String otp, String clientId, int tokenExpiration, @Nullable String user, @Nullable Map<String, Integer> grants)

def ListenableFuture<String> webhookAuth(String authRef)

def ListenableFuture<ClientListReply> getClientList(ClientListRequest it)

def ListenableFuture<UserWriteReply> userWrite(UserWriteRequest it)

def ListenableFuture<UserSearchReply> userSearch(UserSearchRequest it)

def ListenableFuture<UserGroupWriteReply> groupWrite(UserGroupWriteRequest it)

def ListenableFuture<UserGroupSearchReply> groupSearch(UserGroupSearchRequest it)

def ListenableFuture<Boolean> hasAnyPermission(String userName, int permission, String... keys)

def ListenableFuture<Boolean> isMemberOf(String userName, String groupName)

def ListenableFuture<UserConfigWriteReply> configWrite(UserConfigWriteRequest it)

def ListenableFuture<UserConfigSearchReply> configSearch(UserConfigSearchRequest it)

Configuration:

TODO

Usage:

  1. Token from another service
TODO
  1. Verification from another service
TODO
  1. Webhook from another service
TODO
WatchdogService.xtend

Can report runtime health/metrics to another meta-engine runtime, check if other runtimes are healthy and schedule automated updates.

Interface: None

Configuration:

TODO

In addition to these vital services, there are WebSocket and REST-like server APIs for clients:

EngineCore.xtend

Service facade for the WebSocket API. Delegates access control for service calls and returned entities to UserService.

EngineResource.xtend

Provides the REST-like API.

  • EngineAuthResource - OTP webhook authentication, delegates to UserService
  • EngineTokenResource - Issues tokens, delegates to UserService
  • EngineUploadResource - File upload, delegates to FileService
  • EngineDownloadResource - File download, delegates to FileService
EngineWebSocket.xtend

Provides the WebSocket API in the form of a WebSocketListener.

Sends/receives Event objects serialized to JSON over sendString/onWebSocketText.

Sends/receives byte[] file content for DownloadRequest/UploadRequest over sendBytes/onWebSocketBinary.


The actual runtime and other lifecycle functions are provided by these components:

Engine.xtend

Launcher and dependency injection module/components.

EngineRuntime.xtend

Provides the runtime with executor services, ServiceManager for service lifecycle and centralized Metric/Health-Check/Alert registries.

Implements Executor interface and also provides convenience methods to submit and handle async tasks, see ListenableFuture.

Interface:

def boolean awaitDependentServices(Service... services)

def <T> ListenableFuture<T> submit(Function0<T> func)

def void submitVoid(Procedure0 proc)

def ListenableScheduledFuture<?> schedule(Procedure0 proc, long delay, TimeUnit unit)

def ListenableScheduledFuture<?> scheduleWithFixedDelay(Procedure0 proc, long initialDelay, long delay, TimeUnit unit)

def <V> void addCallback(ListenableFuture<V> future, FunctionalFutureCallback<? super V> callback)

def boolean isHealthy()

def boolean isServiceManagerHealthy()

def List<String> getUnhealthyServices()

public MetricRegistry getMetricRegistry()

public HealthCheckRegistry getHealthCheckRegistry()

public AlertRegistry getAlertRegistry()

public boolean isRunning()

Configuration:

TODO

Usage:

  1. Async tasks from another service
val request = SearchRequest.forge(metaId, 1)
val metaFuture = Futures.transform(metaService.search(request), [ searchReply |
    if(searchReply.metas.empty === false) {
        return searchReply.metas.get(0) as MetaObject
    }
    return null
], runtime)
runtime.addCallback(metaFuture, new FunctionalFutureCallback([ exc |
    LOG.warn("metaFuture failed", exc)
], [ meta |
    LOG.info("meta={}", meta)
]))
  1. Alerts from another service
TODO
EngineUpdater.xtend

Execute tasks against two meta-engine instances for importing/exporting and migrating data. The current instance acts as the source and a new, pristine instance acts as the target. After one or more tasks run without error, all data directories of the first, existing instance are replaced with the data directories of the new instance.

Predefined tasks:

  • CloneTask - Clones all data (FileService, MetaService, UserService) to the new instance. This means a complete re-index of the underlying Lucene indexes. Useful in case of schema changes or new Lucene versions
  • EngineImportTask - Imports selected data from a remote engine to the new instance. See EngineImportTaskConfig for configuration options
  • JsonImportTask - Imports all data of the input JSON file to the new instance. See JsonImportTaskFile for file structure
  • JsonExportTask - Exports selected data from the existing instance to a JSON file. The JSON file is in the format of JsonImportTaskFile. See JsonExportTaskConfig for configuration options

Important notes:

  • Tasks are executed in the order that they were passed in
  • If any of the tasks returns true for skipIndexReplacement, the data directories of the existing instance are not replaced with the new ones
  • Replacement of data directories always takes place if all services returns false for skipIndexReplacement. If no task copies or imports any data into the new instance, you end up with an empty instance!
  • To import new data into an existing instance, run clone followed by jsonimport (or other way around if the imported data takes precedence)
  • There is no automatic backup before running the tasks. Use meta-admin to manually create backups

Interface: None

Configuration: None

Usage:

  1. Define a new tasks extending AbstractEngineUpdaterTask
class SimpleEchoTask extends AbstractEngineUpdaterTask {
    val static LOG = LoggerFactory.getLogger(SimpleEchoTask)
    val public static TASK_ID = "echo"

    override getId() {
        return TASK_ID
    }

    override update(EngineUpdaterServices oldServices, EngineUpdaterServices newServices) {
        val searchRequest = SearchRequest.forge('''«Model.ECHO_ACTION_ID»''', 1)
        val searchReply = newServices.metaService.search(searchRequest).get
        if (searchReply.metas.empty === false) {
            LOG.info("echo action exists in new index!")
        }
    }
}
  1. Update entities after changes to the schema with a EngineUpdaterEntityHandler:
class SimpleFieldUpdater implements EngineUpdaterEntityHandler {
    val static LOG = LoggerFactory.getLogger(SimpleFieldUpdater)
    val static OLD_FIELD = "\"oldField\":"
    val static NEW_FIELD = "\"newField\":"

    override handles(Object entity) {
        if(entity instanceof MetaObject) {
            return Model.SIMPLE_TYPE_ID == entity.typeId
        }
        return false
    }

    override <T> update(T entity) {
        val meta = entity as MetaObject
        val valueJson = Cst.OM.writeValueAsString(meta.value)
        val newValueJson = valueJson.replaceAll(OLD_FIELD, NEW_FIELD)
        LOG.info("update, metaId={}, newValueJson={}", meta.id, newValueJson)
        return MetaObject.build(meta, [
            value = Optional.of(JsonizedExtensions.forgeFromString(Cst.OM, newValueJson))
        ]) as T
    }
}
  1. Custom tasks and entity handlers have to be registered in the EngineUpdaterModule:
@Data class MyAppLauncher extends AbstractEngineLauncher<MyAppConfig> {
    override launchUpdater() {
        return DaggerHelper.forgeEngineUpdater(new MyAppEngineUpdaterModule(config.loadEngineConfig))
    }

    override launchRuntime() {
        return DaggerHelper.forgeEngineRuntime(new MyAppConfigModule(config, configPath))
    }
}

@Module class MyAppEngineUpdaterModule extends EngineUpdaterModule {
    new(EngineConfig config) {
        super(config)
    }

    @Provides @IntoSet def EngineUpdaterTask getSimpleEchoTask() {
        return new SimpleEchoTask
    }

    @Provides @IntoSet def EngineUpdaterEntityHandler getSimpleFieldHandler() {
        return new SimpleFieldUpdater
    }
}
HttpService.xtend

Wrapper around a Jetty Server instance with options for SSL, Caching, CORS, CSP, black-/whitelists, rate limits, etc.

Example for a REST resource:

@Path("greeting")
@Produces(MediaType.TEXT_PLAIN)
class GreetingResource {
    val static LOG = LoggerFactory.getLogger(GreetingResource)

    EngineRuntime runtime
    GreetingResourceConfig config

    @Inject new(EngineRuntime runtime, GreetingResourceConfig config) {
        this.runtime = runtime
        this.config = config
    }

    @Path("/hello") @GET
    def void getGreeting(@QueryParam("name") String name) {
        LOG.info("getGreeting, name={}", name)
        async.setTimeout(10, TimeUnit.SECONDS)
        runtime.execute[
            async.resume('''«config.greeting» «name»!''')
        ]
    }
}

@Jsonized @Data class GreetingResourceConfig {
    String greeting
}

@Module class GreetingResourceModule {
    @Provides @IntoSet def HttpServiceRestComponentConfig getGreetingComponent(EngineRuntime runtime, GreetingResourceConfig config) {
        val binder = new AbstractBinder {
            override protected configure() {
                bind(runtime).to(EngineRuntime)
                bind(config).to(GreetingResourceConfig)
            }
        }
        return new HttpServiceRestComponentConfig(#[GreetingResource], binder)
    }
}
Main.xtend

Generic main entry point for all meta-engine based applications. Applications define their own configuration and launcher classes that they pass as arguments.

Interface:

def static void main(String[] args)

Configuration: None

Usage:

  1. Start the instance
java -cp engine.jar ch.rswk.meta.engine.Main -configcls ch.rswk.meta.engine.EngineConfig -config engine.json -launchcls ch.rswk.meta.engine.EngineLauncher -start
  1. Run update tasks
java -cp engine.jar ch.rswk.meta.engine.Main -configcls ch.rswk.meta.engine.EngineConfig -config engine.json -launchcls ch.rswk.meta.engine.EngineLauncher -update clone,jsonimport fixture.json

For more command line examples see:

  • platform/bootstrap/bootstrap-common/src/main/resources/bootstrap/definitions/engine-clone.json
  • platform/bootstrap/bootstrap-common/src/main/resources/bootstrap/definitions/engine-import.json
  • platform/bootstrap/bootstrap-common/src/main/resources/bootstrap/definitions/engine-install-service.json

meta-engine-test

Test module for meta-core, meta-client, meta-engine.

GatewayServiceTest.xtend

Requires custom VM arguments in Eclipse run configuration:

-Dgatewayservicetest.sendername=
-Dgatewayservicetest.sms.user=
-Dgatewayservicetest.sms.httppassword=
-Dgatewayservicetest.sms.gatewaypassword=
-Dgatewayservicetest.email.server=
-Dgatewayservicetest.email.user=
-Dgatewayservicetest.email.password=
-Dgatewayservicetest.email.sender=

meta-admin

Management and monitoring application for meta-engine.

  • Editor - CRUD operations for meta model entities
  • Clients - List all currently connected WebSocket clients
  • Notifications - List and send notifications
  • Users - CRUD operations for all user related entities
  • Logging - Centralied logging and metrics
  • File - CRUD operations for files
  • Watchdog - Check on other engines, schedule local and remote updates
  • Map - Show geospatial data and entities on OSM based map layers

Can be used as reference on how to structure and build web application based on meta.

Editor Clients Notifications Users Logging File Watchdog

meta-test

Test module for meta-core, meta-client and meta-engine.

Util / Web

util-macro

Due to technical restrictions, util-macro contains all Xtend Active Annotations so we can use them in util-core.

Defines the basic Maven dependencies that are inherited by all modules.

Jsonized.xtend

TODO

MutableBuilder.xtend

TODO

Observable.xtend

TODO

Strings.xtend

TODO

util-macro-test

Test module for util-macro.

Derfines the basic Maven dependencies that are inherited by all test modules.

Important Note: Test modules are just regular Maven modules and seperated from the modules under test. This does not conform to the normal way to setup tests with Maven, but we feel like the benefits from transitive dependencies for test modules make up for it.

util-core

Utility functions for server modules.

Env.xtend

TODO

Files.xtend

TODO

FutureCallback.xtend

TODO

P7zip.xtend

TODO

VariableResolver.xtend

TODO

WeakCrypto.xtend

TODO

Wget.xtend

TODO

util-core-test

Test module for util-core-test.

web-core

Utility functions for client modules.

Defines the build process for web clients:

  • web-build.bat or web-build.sh is called from Maven build
  • myapp-web/src/main/resources/web-build.json
    • Configuration for web-build.js build script
  • myapp-web/src/main/resources/web-dist.js
    • Configuration for r.js bundler/optimizer
  • myapp-web/src/main/resources/web-custom.js
    • Any custom Node script for module specific needs
checker.js

Check if current browser supports a feature/API and generate log entries if not.

checker.require("navigator.serviceWorker", navigator.serviceWorker);
console.log(checker.isMissingFeatures());
ConfigLoader.xtend / configloader.js

RequireJS plugin to overwrite specific properties from a config object. The config overwrite is passed as a query string.

  1. In Java test
val configOverwrite = forgeFromString(OM, '''{"debug":true}''')
val path = '''myapp-web/src/main/resources/?«ConfigLoader.forgeQueryString(OM, configOverwrite)»'''
val uri = forgePlatformUri(HTTP_SERVICE_TEST_PORT, path).toString
  1. In JavaScript
define([
    "web-core/configloader!mycompany-web/config", "domReady!"
], function (config) {..});
crypto.js

Possibly unsafe crypto functions using crypto and crypto.subtle.

let id = crypto.generateRandomString();

Can be used to generate, export, import keys and then sign, verify, encrypt and decrypt (see crypto-test.js).

Device.xtend / device.js

TODO: Explain how device works and what it can do

Implementations for browser and Android

  • device-android.js
  • device-browser.js
ext.js

Various extension and helper functions.

  1. Key-Value Store
ext.db.set("key", {value: 1}).then(function () {
    return ext.db.get("key");
}).then(function (obj) {
    console.log(obj.value);
    return ext.db.delete("key");
});
i18n.js

Wrapper for i18next so it can be used in a Knockout application.

  1. In i18nResourceStore.js
define(["myapp-web/config", "meta-client/i18nResourceStore", "web-core/i18n"], function (config, clientResourceStore, i18n) {
    "use strict";

    return {
        de: i18n.forgeExtendedResourceStore("de", clientResourceStore, config, {
            greeting: "Hallo {{name}}",
            title: "Begrüssung"
        }),
        en: i18n.forgeExtendedResourceStore("en", clientResourceStore, config, {
            greeting: "Hallo {{name}}",
            title: "Greeting"
        })
    };
});
  1. In JavaScript
const title = i18n.t("title");
const greetingComputed = i18n.pureComputedT("greeting", {name: "World"});
  1. In HTML
<h1 data-bind="i18n: 'title'"></h1>
<span data-bind="i18n: {key: 'greeting', options: {name: 'World'}}"></span>
nav.js
// Register with nav
nav.register("mycomponent", function onUpdate(optionalParam) {
    console.log(`optionalParam=${optionalParam}`);
}, function onRemove() {
    console.log("Bye!");
});

// Navigate to component
const params = nav.newParams(); // or nav.currentParams() to only overwrite certain parameters
params.set("active_component", "mycomponent");
nav.pushState(params);

// or nav.pushState(params, {...}) to pass more application state
// or nav.replaceState(...) to replace the current history entry
ui.js

UI extensions, automatically includes Bootstrap JavaScript modules.

Contains custom Knockout binding handlers:

  • busyButton

Shows a spinner/disables button when the value is true.

  1. In HTML
<form data-bind="submit: load">
    <button type="submit" class="btn btn-primary" data-bind="busyButton: isLoading">
        <span data-bind="i18n: 'search'"></span>
    </button>
</form>

Optional second binding buttonDisabled that will disable the button if value is true and will play nice with the busy indicator.

  1. In JavaScript
that.isLoading = ko.observable();
that.load = function () {
    that.isLoading(true);
    return doTheLoading().then(
        that.isLoading(false);
    });
};
  • fadeIn
  • fadeOut
  • fixHeight

Sets the elements height according to the current window height. Updates when ever the value changes or the window is resized.

  1. In HTML
<div class="row" data-bind="fixHeight: isActive">
    ...
</div>
  • modal

Shows a modal if the value is true and hides it if the value is false.

  1. In HTML
<div class="modal" tabindex="-1" role="dialog" data-bind="modal: activeObject">
    <div class="modal-dialog" role="document" data-bind="with: activeObject">
        ...
    </div>
</div>

Optional second binding confirmClose that will show a confirmation dialog (text of dialog must be set with confirmCloseI18n) if the modal is about to be closed and the value of the binding returns true.

  1. In JavaScript
that.activeObject = ko.observable();
that.editObject = function (object) {
    that.activeObject(object);
};
worker.js
  1. In JavaScript with a worker-init.js (See meta-admin for an example)
define(["myapp-web/config", "web-core/worker"], function (config, worker) {
    "use strict";

    return worker({
        name: "myapp-web",
        version: config.version,
        debug: config.debug
    });
});

web-core-test

Test module for web-core.

Defines AbstractWebDriver that is used for all Selenium based UI tests, including running and asserting all QUnit client side tests.

Android

Android WebView based wrapper to enhance wep applications with access to hardware specific functions of a smartphone, like Bluetooth or NFC.

Web applications must use device module from web-core, where the API between the two is defined.

This wrapper becomes obsolete, once Web-APIs provide the remaining functions that are currently only possible with native functions.

IDE

Custom Eclipse based IDE for Platform developers. Is skipped in the regular Maven build due to the time it takes. Can be run manually (eg. to update to latest Eclipse version) and creates *nix and Windows based products.

The final product ZIPs can be found in platform/ide/ch.rswk.ide.product/target\products.

Bootstrap (Framework)

Small framework that works on *nix and Windows to:

  • Create simple automated installers, called bootstrap modules
  • Create customer/service specific versions of these bootstrap modules
  • Provision servers with the bootstrap modules
  • Execute bootstrap modules

bootstrap-core

Main components to build, provision and execute bootstrap modules.

Build.xtend

TODO

Provision.xtend

TODO

Execute.xtend

TODO

Built in actions:

  • CmdActionExecutor.xtend - Execute shell command
  • CopyActionExecutor.xtend - Copy file or folders
  • EnvvarActionExecutor.xtend - Add, append or remove environment variables
  • IncludeActionExecutor.xtend - Include other bootstrap definitions
  • RestActionExecutor.xtend - Execute HTTP requests against REST-like APIs
  • UnpackActionExecutor.xtend - Unpack archives
  • WgetActionExecutor.xtend - Download files via HTTP

Every action supports conditions for its execution:

  • env - Only execute if running on particular OS
  • exists - Only execute if this file/folder exists or if prefixed with ! not exists
  • flag - Only execute if this flag is present or if prefixed with ! not present
Main.xtend

Main class to run the three phases with commands:

  • -build
  • -execute
  • -provision

bootstrap-module

CLI scripts to bootstrap and run bootstrap-core on a PC/server. Defines the Maven assembly for the specific file structure of a bootstrap module:

- bootstrap-my-server
    - src/main/bootstrap
        - bin // Binary dependencies. Do not put stuff here manually
        - config // Configuration files. Maven filtering is applied to these during build
            - engine.json
        - definitions // JSON bootstrap definitions. Maven filtering is applied here as well
        - fixture // Any file that should not be touched by Maven filterig
    - pom.xml

Binary dependencies for the bin directory can be defined in the Maven build using maven-dependency-plugin:

<build>
    <plugins>
        <plugin>
            <artifactId>maven-dependency-plugin</artifactId>
            <executions>
                <execution>
                    <phase>prepare-package</phase>
                    <goals>
                        <goal>copy</goal>
                    </goals>
                    <configuration>
                        <artifactItems>
                            <!-- Web -->
                            <artifactItem>
                                <groupId>ch.rswk.platform</groupId>
                                <artifactId>meta-admin</artifactId>
                                <version>${project.version}</version>
                                <type>zip</type>
                                <overWrite>true</overWrite>
                                <outputDirectory>${bootstrapBinDir}</outputDirectory>
                            </artifactItem>
                            <!-- Engine -->
                            <artifactItem>
                                <groupId>ch.rswk.platform</groupId>
                                <artifactId>meta-engine</artifactId>
                                <version>${project.version}</version>
                                <type>jar</type>
                                <classifier>shaded</classifier>
                                <overWrite>true</overWrite>
                                <outputDirectory>${bootstrapBinDir}</outputDirectory>
                            </artifactItem>
                        </artifactItems>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

The contents of other bootstrap modules can be included by adding them as a Maven dependency:

<dependencies>
    <dependency>
        <groupId>ch.rswk.platform.bootstrap</groupId>
        <artifactId>bootstrap-common</artifactId>
        <version>${project.version}</version>
        <type>zip</type>
    </dependency>
</dependencies>

This general folder structure is present in the regular source repository, eg. platform, where the base version of a bootstrap module is defined. Any number of customer/service specific versions can be defined in a seperate bootstrap source repositor, eg. platform-bootstrap. This is also where the build or provision specific config files are usually stored.

- customerone
    - bootstrap-watchdog-server
        - config
            - engine.json // Overwrite options from base module engine.json
        - build.json // Configuration for bootstrap-core build step
        - provision.json // Configuration for bootstrap-core provision step

For example see platform/bootstrap/bootstrap-watchdog-server for a basic module and platform-bootstrap/customerone/bootstrap-watchdog-server for a customer specific module that overwrites a couple of files and config options. The modules are merged during the build process:

SET CUSTOMER_DIR=platform-bootstrap\customerone\bootstrap-watchdog-server
java -jar bootstrap.jar -build %CUSTOMER_DIR%\build.json -module platform\bootstrap\bootstrap-watchdog-server -customer %CUSTOMER_DIR% -version %VERSION%

Variables for the build step can be defined with:

  • properties in the JSON configuration of the build step (1)
    • Two predefined variables are also available during build
      • moduleDir and customerDir
  • Java System Properties form the command line, eg. with -DmyProperty=123 (2)
  • Environment variables, depending on the OS (3)

All variables for the build step can be used:

  • Inside the JSON configuration with {myProperty}
  • Inside any files in config and definitions folder with Maven Filtering
    • (1)/(2) with ${myProperty}
    • (3) with ${env.myProperty}

The build step generates the bootstrap.json that is the configuration file for the execution step of bootstrap-core.

Variables for the execution step can be defined with:

  • properties in the JSON configuration of the execution step (1)
    • All properties of JSON configuration of the build step are added automatically
    • A couple of predefined variables are also available during execution
      • {moduleDir}
      • {dataDir}, {logDir}, {toolDir}
        • Including a Windows specific version, eg. {dataDirWin}
      • {JAVA_HOME}
  • Java System Properties from the comman dline, eg. with -DmyProperty=123 (2)
  • Environment variables, depending on the OS (3)

All variables for the execution step can be used in the bootstrap JSON definitions with {myProperty}.

{
    "actions": [
        {
            "wget": {
                "uri": "https://mycompany.ch/myfile.zip",
                "target": "{moduleDir}/download/myfile.zip"
            }
        },
        {
            "cmd": {
                "command": "systemctl enable {serviceName}"
            }
        }
    ]
}

bootstrap-test (Framework)

Test module for bootstrap-core.

Defines AbstractBootstrapTest that is used for testing bootstrap modules by building, provisioning servers for *nix and Windows and executing the modules.

ProvisionTest.xtend

Requires custom VM arguments in Eclipse run configuration:

-Dprovisiontest.watchdog.password=
-Dprovisiontest.vultr.apikey=
-Dprovisiontest.vultr.serverpassword=
-Dprovisiontest.vultr.reservedip=
-Dprovisiontest.vultr.firewallgroup=

Bootstrap

bootstrap-dev-vm

Bootstrap module to provision the development environment for working on the Platform.

bootstrap-dev-server

Bootstrap module to provision the development server of the Platform.

bootstrap-watchdog-server

Bootstrap module to provision a watchdog server.

Can be used as reference to provision meta-engine based servers.

bootstrap-test

Test module for all bootstrap modules.

BootstrapTest.xtend

Requires custom VM arguments in Eclipse run configuration:

-Dprovisiontest.watchdog.password=
-Dprovisiontest.vultr.apikey=
-Dprovisiontest.vultr.serverpassword=
-Dprovisiontest.vultr.reservedip=
-Dprovisiontest.vultr.firewallgroup=

Use

Use meta-engine as a standalone server component. Create simple meta models using meta-admin as a client and explore all the basic Platform functionality.

The scripts below are made to run in Ubuntu and Windows. Should also work on other *nix and *BSD variants, but may need some changes. On Windows, run in PowerShell as Administrator.

If you already have a local JDK/JRE (Java 11 or newer), skip the first step.

1. Download and extract the JDK

Ubuntu:

cd /var/tmp
wget -O java13.tar.gz "https://cdn.azul.com/zulu/bin/zulu13.28.11-ca-jdk13.0.1-linux_x64.tar.gz"
tar -zxf java13.tar.gz
JRE=$PWD/zulu13.28.11-ca-jdk13.0.1-linux_x64/bin

Windows:

$global:ProgressPreference = 'SilentlyContinue'
cd C:\tmp
Invoke-WebRequest -Uri https://cdn.azul.com/zulu/bin/zulu13.28.11-ca-jdk13.0.1-win_x64.zip -OutFile java13.zip
Expand-Archive -LiteralPath java13.zip
$JRE = (Get-Location).tostring() + "\zulu13.28.11-ca-jdk13.0.1-win_x64\bin"

2. Download server and client

Ubuntu:

VERSION='0.1106'
wget -O engine.jar "https://dev.platform.rswk.ch/nexus/repository/maven-releases/ch/rswk/platform/meta-engine/$VERSION/meta-engine-$VERSION-shaded.jar"
wget -O admin.zip "https://dev.platform.rswk.ch/nexus/repository/maven-releases/ch/rswk/platform/meta-admin/$VERSION/meta-admin-$VERSION.zip"
sudo apt-get install unzip
unzip admin.zip -d admin

Windows:

$VERSION = '0.1106'
Invoke-WebRequest -Uri "https://dev.platform.rswk.ch/nexus/repository/maven-releases/ch/rswk/platform/meta-engine/$VERSION/meta-engine-$VERSION-shaded.jar" -OutFile engine.jar
Invoke-WebRequest -Uri "https://dev.platform.rswk.ch/nexus/repository/maven-releases/ch/rswk/platform/meta-admin/$VERSION/meta-admin-$VERSION.zip" -OutFile admin.zip
Expand-Archive -LiteralPath admin.zip -DestinationPath admin

3. Setup localhost keystore

Ubuntu:

wget -O test.jar "https://dev.platform.rswk.ch/nexus/repository/maven-releases/ch/rswk/platform/web-core-test/$VERSION/web-core-test-$VERSION.jar"
"$JRE/jar" xf test.jar localhost.platform.rswk.ch.pfx

Windows:

Invoke-WebRequest -Uri "https://dev.platform.rswk.ch/nexus/repository/maven-releases/ch/rswk/platform/web-core-test/$VERSION/web-core-test-$VERSION.jar" -OutFile test.jar
& "$JRE\jar.exe" xf test.jar 'localhost.platform.rswk.ch.pfx'

4. Manual steps

  • Change admin/js/config.js
    • "workerPath: "js/"
    • "engine":"localhost.platform.rswk.ch:9090"
  • Add to hosts (/etc/hosts or C:/Windows/System32/drivers/etc/hosts)
    • 127.0.0.1 localhost.platform.rswk.ch

5. Import test user and start server

Ubuntu:

CONFIG=$(cat <<EOF
{
    "action": {},
    "database": {
        "checkoutTimeout": 1000,
        "dbs": []
    },
    "runtime": {
        "startUpTimeout": 10,
        "shutDownTimeout": 10
    },
    "file": {
        "storageDir": "file/store",
        "cleanupSchedule": 300,
        "bufferSize": 0,
        "lucene": {
            "indexDir": "file/lucene",
            "analyzer": null,
            "commitSchedule": 300,
            "workQueue": {
                "capacity": 1000,
                "offerTimeout": 2000
            }
        }
    },
    "gateway": {
        "smsUser": "",
        "smsHttpPassword": "",
        "smsGatewayPassword": "",
        "smsSender": "",
        "emailServer": "",
        "emailPort": 465,
        "emailSsl": true,
        "emailTls": false,
        "emailUser": "",
        "emailPassword": "",
        "emailSender": ""
    },
    "http": {
        "serverDomain": "localhost.platform.rswk.ch:9090",
        "maxThreads": 50,
        "httpPort": 0,
        "httpsPort": 443,
        "ssl": {
            "keystore": "localhost.platform.rswk.ch.pfx",
            "keystorePw": "platform",
            "protocols": [
                "TLSv1.2"
            ],
            "cipherSuites": [
                "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
                "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
                "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
                "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
                "TLS_DHE_RSA_WITH_AES_128_GCM_SHA256",
                "TLS_DHE_RSA_WITH_AES_256_GCM_SHA384",
                "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256",
                "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
                "TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",
                "TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA",
                "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
                "TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384",
                "TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA"
            ]
        },
        "logPath": "logs",
        "resourceBase": "admin",
        "cacheControl": null,
        "header": {
            "cors": null,
            "csp": "connect-src 'self' wss://localhost.platform.rswk.ch:9090 https://*.tile.openstreetmap.org; font-src 'self'; frame-src 'self'; img-src 'self' https://*.tile.openstreetmap.org data:; ​manifest-src 'self'; media-src 'self' mediastream: blob:; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'unsafe-inline' 'self' data:; worker-src 'self' blob:"
        },
        "access": null,
        "maxTextMessageSize": 0,
        "maxTextMessageBufferSize": 0,
        "maxBinaryMessageSize": 0,
        "maxBinaryMessageBufferSize": 0,
        "webSocketRateLimit": 0
    },
    "logging": {
        "monitorSchedule": 30,
        "retention": 7,
        "loggers": null,
        "alertSmsReceiver": null,
        "alertSmsMessage": "WatchdogAlert",
        "alertEmailReceiver": null,
        "alertEmailSubject": "WatchdogAlert",
        "lucene": {
            "indexDir": "logging/lucene",
            "analyzer": null,
            "commitSchedule": 0,
            "workQueue": {
                "capacity": 1000,
                "offerTimeout": 2000
            }
        }
    },
    "meta": {
        "lucene": {
            "indexDir": "meta/lucene",
            "analyzer": null,
            "commitSchedule": 300,
            "workQueue": {
                "capacity": 1000,
                "offerTimeout": 2000
            }
        }
    },
    "notification": {
        "cacheCapacity": 1000
    },
    "sync": {
        "engineId": "localhost",
        "peers": null,
        "etcd": null
    },
    "user": {
        "otpEmailSubject": "YourOTP",
        "authTimeout": 60,
        "attempts": 5,
        "scryptCpu": 1024,
        "scryptMemory": 1,
        "token": null,
        "lucene": {
            "indexDir": "user/lucene",
            "analyzer": null,
            "commitSchedule": 300,
            "workQueue": {
                "capacity": 1000,
                "offerTimeout": 2000
            }
        }
    },
    "watchdog": {
        "report": null,
        "update": null,
        "check": null
    }
}
EOF
)
FIXTURE=$(cat <<EOF
{
    "users": [
        {
            "name": "test",
            "password": "test",
            "grants": {
                "cl": 1,
                "dl": 1,
                "ac": 1,
                "fs": 1,
                "li": 1,
                "nc": 1,
                "cln": 1,
                "men": 1,
                "se": 1,
                "sy": 1,
                "tk": 1,
                "up": 1,
                "wr": 1
            }
        }
    ],
    "groups": [
        {
            "name": "admin",
            "users": [
                "test"
            ],
            "grants": {
                "egn": 3,
                "wsst": 15
            }
        }
    ]
}
EOF
)
"$JRE/java" -Xms128m -Xmx1g -DPLATFORM_LOG_DIR=logs -cp "engine.jar" 'ch.rswk.meta.engine.Main' -configcls 'ch.rswk.meta.engine.EngineConfig' -config "$CONFIG" -launchcls 'ch.rswk.meta.engine.EngineLauncher' -update clone,jsonimport $FIXTURE
"$JRE/java" -Xms128m -Xmx1g -DPLATFORM_LOG_DIR=logs -cp "engine.jar" 'ch.rswk.meta.engine.Main' -configcls 'ch.rswk.meta.engine.EngineConfig' -config "$CONFIG" -launchcls 'ch.rswk.meta.engine.EngineLauncher' -start

Windows:

$CONFIG = @"
{
    \"action\": {},
    \"database\": {
        \"checkoutTimeout\": 1000,
        \"dbs\": []
    },
    \"runtime\": {
        \"startUpTimeout\": 10,
        \"shutDownTimeout\": 10
    },
    \"file\": {
        \"storageDir\": \"file/store\",
        \"cleanupSchedule\": 300,
        \"bufferSize\": 0,
        \"lucene\": {
            \"indexDir\": \"file/lucene\",
            \"analyzer\": null,
            \"commitSchedule\": 300,
            \"workQueue\": {
                \"capacity\": 1000,
                \"offerTimeout\": 2000
            }
        }
    },
    \"gateway\": {
        \"smsUser\": \"\",
        \"smsHttpPassword\": \"\",
        \"smsGatewayPassword\": \"\",
        \"smsSender\": \"\",
        \"emailServer\": \"\",
        \"emailPort\": 465,
        \"emailSsl\": true,
        \"emailTls\": false,
        \"emailUser\": \"\",
        \"emailPassword\": \"\",
        \"emailSender\": \"\"
    },
    \"http\": {
        \"serverDomain\": \"localhost.platform.rswk.ch:9090\",
        \"maxThreads\": 50,
        \"httpPort\": 0,
        \"httpsPort\": 443,
        \"ssl\": {
            \"keystore\": \"localhost.platform.rswk.ch.pfx\",
            \"keystorePw\": \"platform\",
            \"protocols\": [
                \"TLSv1.2\"
            ],
            \"cipherSuites\": [
                \"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\",
                \"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256\",
                \"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\",
                \"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\",
                \"TLS_DHE_RSA_WITH_AES_128_GCM_SHA256\",
                \"TLS_DHE_RSA_WITH_AES_256_GCM_SHA384\",
                \"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256\",
                \"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256\",
                \"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA\",
                \"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA\",
                \"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384\",
                \"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384\",
                \"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA\"
            ]
        },
        \"logPath\": \"logs\",
        \"resourceBase\": \"admin\",
        \"cacheControl\": null,
        \"header\": {
            \"cors\": null,
            \"csp\": \"connect-src 'self' wss://localhost.platform.rswk.ch:9090 https://*.tile.openstreetmap.org; font-src 'self'; frame-src 'self'; img-src 'self' https://*.tile.openstreetmap.org data:; ​manifest-src 'self'; media-src 'self' mediastream: blob:; script-src 'self' 'unsafe-eval' 'unsafe-inline'; style-src 'unsafe-inline' 'self' data:; worker-src 'self' blob:\"
        },
        \"access\": null,
        \"maxTextMessageSize\": 0,
        \"maxTextMessageBufferSize\": 0,
        \"maxBinaryMessageSize\": 0,
        \"maxBinaryMessageBufferSize\": 0,
        \"webSocketRateLimit\": 0
    },
    \"logging\": {
        \"monitorSchedule\": 30,
        \"retention\": 7,
        \"loggers\": null,
        \"alertSmsReceiver\": null,
        \"alertSmsMessage\": \"WatchdogAlert\",
        \"alertEmailReceiver\": null,
        \"alertEmailSubject\": \"WatchdogAlert\",
        \"lucene\": {
            \"indexDir\": \"logging/lucene\",
            \"analyzer\": null,
            \"commitSchedule\": 0,
            \"workQueue\": {
                \"capacity\": 1000,
                \"offerTimeout\": 2000
            }
        }
    },
    \"meta\": {
        \"lucene\": {
            \"indexDir\": \"meta/lucene\",
            \"analyzer\": null,
            \"commitSchedule\": 300,
            \"workQueue\": {
                \"capacity\": 1000,
                \"offerTimeout\": 2000
            }
        }
    },
    \"notification\": {
        \"cacheCapacity\": 1000
    },
    \"sync\": {
        \"engineId\": \"localhost\",
        \"peers\": null,
        \"etcd\": null
    },
    \"user\": {
        \"otpEmailSubject\": \"YourOTP\",
        \"authTimeout\": 60,
        \"attempts\": 5,
        \"scryptCpu\": 1024,
        \"scryptMemory\": 1,
        \"token\": null,
        \"lucene\": {
            \"indexDir\": \"user/lucene\",
            \"analyzer\": null,
            \"commitSchedule\": 300,
            \"workQueue\": {
                \"capacity\": 1000,
                \"offerTimeout\": 2000
            }
        }
    },
    \"watchdog\": {
        \"report\": null,
        \"update\": null,
        \"check\": null
    }
}
"@
$FIXTURE = @"
{
    \"users\": [
        {
            \"name\": \"test\",
            \"password\": \"test\",
            \"grants\": {
                \"cl\": 1,
                \"dl\": 1,
                \"ac\": 1,
                \"fs\": 1,
                \"li\": 1,
                \"nc\": 1,
                \"cln\": 1,
                \"men\": 1,
                \"se\": 1,
                \"sy\": 1,
                \"tk\": 1,
                \"up\": 1,
                \"wr\": 1
            }
        }
    ],
    \"groups\": [
        {
            \"name\": \"admin\",
            \"users\": [
                \"test\"
            ],
            \"grants\": {
                \"egn\": 3,
                \"wsst\": 15
            }
        }
    ]
}
"@
& "$JRE\java.exe" -Xms128m -Xmx1g -DPLATFORM_LOG_DIR=logs -cp "engine.jar" 'ch.rswk.meta.engine.Main' -configcls 'ch.rswk.meta.engine.EngineConfig' -config $CONFIG -launchcls 'ch.rswk.meta.engine.EngineLauncher' -update clone,jsonimport $FIXTURE
& "$JRE\java.exe" -Xms128m -Xmx1g -DPLATFORM_LOG_DIR=logs -cp "engine.jar" 'ch.rswk.meta.engine.Main' -configcls 'ch.rswk.meta.engine.EngineConfig' -config $CONFIG -launchcls 'ch.rswk.meta.engine.EngineLauncher' -start

6. Explore meta-admin client

  • https://localhost.platform.rswk.ch:9090/
    • User test
    • Password test

TODO: Create type, object, etc.

Extend

Extend meta and other parts of the Platform to build more complex meta models and applications with custom backend services. Setup a source repository based on platform-archetype/platform-archetype-bootstrap and inherit all the required settings from the Platform parent POM.

Overview of the whole setup with company specific development server, development workstation, watchdog and application servers. Step by step guide on how to setup all of this below.

TODO: Expand on how these things work together and why we need all of them

Development environment

Production environment

Bootstrap dev-vm

1. Setup and install VM

Setup a Ubuntu or Windows VM with a user account dev, eg. using VirtualBox. If you are using another *nix or *BSD variant, some details may need to be changed. Use scripts below to start the bootstrap process on your new VM.

Ubuntu:

VERSION="0.1106"
cd /var/tmp
wget -O bootstrap.zip "https://dev.platform.rswk.ch/nexus/repository/bootstrap/ch/rswk/platform/bootstrap/bootstrap-dev-vm-platform-nix/$VERSION/bootstrap-dev-vm-platform-nix-$VERSION.zip"
sudo apt-get install unzip
unzip bootstrap.zip -d bootstrap
cd bootstrap
chmod +x fixture/bootstrap.sh
sudo ./fixture/bootstrap.sh -execute $PWD -setowner dev

Windows:

$global:ProgressPreference = 'SilentlyContinue'
$VERSION = '0.1106'
cd C:\tmp
Invoke-WebRequest -Uri https://dev.platform.rswk.ch/nexus/repository/bootstrap/ch/rswk/platform/bootstrap/bootstrap-dev-vm-platform-win/$VERSION/bootstrap-dev-vm-platform-win-$VERSION.zip -OutFile bootstrap.zip
Expand-Archive -LiteralPath bootstrap.zip -DestinationPath bootstrap
cd bootstrap
Set-ExecutionPolicy Unrestricted
.\fixture\bootstrap.ps1 -execute (Get-Location).tostring() -setowner dev

2. Manual steps dev-vm

  • Setup Eclipse
    • Window > Preferences
      • Maven > Installations > Add
        • *nix /usr/share/maven
        • Windows C:\tool\maven
      • Xtend > Compiler > Output folder for generated Java files > ../../target/generated-sources/xtend
      • Xtend > Formatter > New > Initialize settings from “default”
      • Maven > Annotation Processing > Automatically configure
      • General > Editors > Text Editors > Insert spaces for tabs
      • Java > Code Style > Formatter > New
        • Indentation > Tab policy > Spaces only
      • Java > Installed JREs
        • *nix /usr/local/platform/tool/jdk13
        • Windows C:\tool\jdk13
    • Import maven modules
      • File > Import > Existing Maven Projects
        • *nix /usr/local/platform/hg/platform
        • Windows C:\hg\platform
      • Deselect All
      • Select all child modules
        • Eg. for util: util-macro, util-macro-tests, util-core, util-core-tests)
        • Ignore meta-admin (since it’s a pure web module) and all modules under the top level bootstrap module except bootstrap-test
    • Enable Xtend
      • Select all projects in Package Explorer
      • Right click > Configure > Convert to Xtext Project
      • Wait for build, in case of errors Project > Clean > Clean all projects
  • Setup Visual Studio Code
    • Add SCM root to Workspace
      • *nix /usr/local/platform/hg
      • Windows C:\hg
    • Extensions
      • CSS Formatter
      • Hg
      • XML Tools
  • Install and setup Android Studio
    • In downlad directory where you extracted the bootstrap module find
      • *nix android-studio.tar.gz
      • Windows android-studio.exe
    • Install
      • *nix unpack archive to /usr/local/platform/tool/android-studio and run ./studio.sh
      • Windows run installer
    • Setup
      • Do not import settings
      • Custom
      • Android SDK location
        • *nix /usr/local/platform/tool/android-sdk
        • Windows C:\tool\android-sdk
      • Finish
    • Import project
      • *nix /usr/local/platform/hg/platform/android
      • Windows C:\hg\platform\android
  • Optional steps
    • Windows Virus & threat protection
      • Add C:\data, C:\hg, C:\tool to exclusion settings
    • Setup WiFi Hotpost for local testing with mobile devices
      • Windows Settings > Network > Mobile hotspot
        • Set SSID and Password and turn on
        • Change adapter options > Microsoft WiFi Direct Virtual Adapter > Properties > Internet Protocol Version 4
          • Use the following IP: 192.168.1.1
          • Use the following DNS: 192.168.1.1
      • Install and setup Simple DNS Plus
        • In downlad directory where you extracted the bootstrap module find simpledns.exe
          • *nix 🤷‍♂️
          • Windows Run installer
      • Setup
        • Records > New Zone > Primary Zone > Forward Zone > rswk.ch
        • Right click rswk.ch > New A-Record > localhost.platform.rswk.ch with IP 192.168.1.1
      • If you are running in a VM, make sure to forward the necessary ports!
    • Setup em microelectronic Beacons
      • Product https://www.emmicroelectronic.com/catalog?title=&term_node_tid_depth=42
      • Tool kit required https://www.emmicroelectronic.com/product/microcontroller-tools-support/em6819-tool-kit
      • SDK
        • Sign up on https://forums.emdeveloper.com/index.php
        • Download latest https://forums.emdeveloper.com/index.php?resources/embc01-02-sdk.18/
        • Unpack SDK, eg. to C:/tool/EMBCxx
        • Download and install RubyInstaller from https://rubyinstaller.org/downloads/
      • Ride7
        • Download https://dev.platform.rswk.ch/dl/sx32w.dll.zip
          • Copy sx32w.dll to C:\tool\EMBCxx\util\bin
          • Add this folder to PATH
        • Download and insall Ride7 IDE from http://support.raisonance.com/content/ride

3. Optional: Start meta-admin from Eclipse

  • Project meta-admin-test
    • Open WebDriverTest.xtend
      • On the last line of setupRuntime, insert System.in.read
    • Right click on WebDriverTest.xtend > Run As > JUnit Test
    • Open browser https://localhost.platform.rswk.ch:9090/meta/meta-admin/src/main/resources/

Bootstrap dev-server

1. Build dev-server module

  • Build a local copy of the bootstrap-dev-server module, parameterized for your own development server
  • TODO: Explain how to parameterize build
  • Final ZIP is bootstrap/bootstrap-dev-server/target/bootstrap-dev-server-1.0-SNAPSHOT.zip

2. Upload and install dev-server

  • Setup a Ubuntu or Windows Server VM, eg. with a cloud computing provider like Vultr
  • Upload bootstrap-dev-server-1.0-SNAPSHOT.zip as bootstrap.zip to server
    • *nix /var/tmp
    • Windows C:/tmp

Ubuntu:

cd /var/tmp
sudo apt-get install unzip
unzip bootstrap.zip -d bootstrap
cd bootstrap
chmod +x fixture/bootstrap.sh
sudo ./fixture/bootstrap.sh -execute $PWD

Windows:

$global:ProgressPreference = 'SilentlyContinue'
cd C:\tmp
Expand-Archive -LiteralPath bootstrap.zip -DestinationPath bootstrap
cd bootstrap
Set-ExecutionPolicy Unrestricted
.\fixture\bootstrap.ps1 -execute (Get-Location).tostring()

3. Manual steps dev-server

  • Install Android Studio
    • In the bootstrap downlad directory, find
      • *nix android-studio.tar.gz
      • Windows android-studio.exe
    • Install
      • *nix unpack archive to /usr/local/platform/tool/android-studio and run ./studio.sh
      • Windows run installer
    • Setup
      • Do not import settings
      • Custom
      • Android SDK location
        • *nix /usr/local/platform/tool/android-sdk
        • Windows C:\tool\android-sdk
      • Finish
  • Jenkins
    • Install suggested plugins
    • Manage plugins
      • Available
        • Mercurial
        • Maven Integration
    • Global tool configuration
      • JDK
        • *nix /usr/local/platform/tool/java13
        • Windows C:\tool\java13
      • Git
        • *nix 🤷‍♂️
        • Windows C:\tool\git\bin\git.exe
      • Mercurial
        • *nix 🤷‍♂️
        • Windows C:\tool\tortoisehg, INSTALLATION\hg.exe
      • Maven
        • *nix /usr/share/maven
        • Windows C:\tool\maven
    • Credentials > Jenkins > Global credentials
      • Add Credentials
        • platform and password platform
          • scmadmin and password
  • Nexus
    • Change password of admin user
    • Add hosted maven2 repo bootstrap (release, allow redeploy), thirdparty (release)
    • Add hosted raw repo site
    • Add proxy maven2 repo google
    • Add proxy maven2 repo jcenter
    • Add proxy maven2 repo platform
    • Add all to maven-public repo
    • Add cleanup policies
      • Repository > Cleanup Policies > Create
        • Format maven2
        • Last Downloaded Before 200
      • Repository > Repositories
        • bootstrap, maven-central, maven-release (and others if needed)
        • Set cleanup policy
    • Setup compacting task
      • System > Tasks > Create task
      • Compact Blob Store
      • Blob store default
      • Task frequency monthly
      • Time to run this task 2:00
      • Days to run this task Last

4. Setup repositories

  • Login at http://127.0.0.1:8082/scm
  • Create mycompany and mycompany-bootstrap Mercurial repositories
    • Note: This is only required for the initial server installation, it should become part of your dev-server bootstrap definition
  • Create additional SCM users and grant permissions for repos as needed

5. Setup CI and bootstrap jobs

Back on your development VM, open .hgrc and adapt for your development server.

TODO: Explain what to adapt

Clone mycompany-bootstrap:

  • hg clone https://dev.mycompany.ch/scm/hg/mycompany-bootstrap

Copy job configs from platform-archetype-bootstrap/build/jobs to mycompany-bootstrap/build/jobs.

Rename and adapt job configs:

  • mycompany.xml
  • bootstrap-dev-server-mycompany.xml
  • bootstrap-dev-vm-mycompany.xml
  • bootstrap-watchdog-server-mycompany.xml

TODO: Explain how to adapt these jobs

Commit changes in mycompany-bootstrap repository and push to server.

Back on your development server, import all job configs to Jenkins:

  • *nix curl --data-binary @mycompany.xml --request POST --header 'Content-Type: text/xml' 'http://127.0.0.1:8082/jenkins/createItem?name=mycompany'
  • Windows curl -body "file=$(get-content mycompany.xml -raw)" -method POST -ContentType 'text/xml' -uri 'http://127.0.0.1:8082/jenkins/createItem?name=mycompany'

TODO: Copy manually to dev-server?

Change build number if needed in Jekins script console:

  • Jenkins.instance.getItemByFullName("mycompany").updateNextBuildNumber(123)

My Company

1. Populate repository from platform-archetype

Clone main repository to your development VM:

  • hg clone https://dev.mycompany.ch/scm/hg/mycompany

Copy contents of platform-archetype repository into mycompany and adapt.

TODO: Explain what to adapt

Adapt rest of mycompany-bootstrap/mycompany, including build.json files.

TODO: Explain what to adapt

Commit and push mycompany and mycompany-bootstrap to server.

2. Run build jobs

  • mycompany
  • bootstrap-dev-server-mycompany
    • TODO: Params
  • bootstrap-dev-vm-mycompany
    • TODO: Params

Now you can loop back to the first chapter and setup a new development VM, but instead of using the Platform Nexus, use your own Nexus to download the dev-vm bootstrap module:

  • https://dev.mycompany.ch/nexus/repository/bootstrap/ch/mycompany/bootstrap/bootstrap-dev-vm-mycompany/$VERSION/bootstrap-dev-vm-mycompany-$VERSION.zip

With the new VM, all your company repositories should already be checked out and the Maven build should run off your Nexus server.

3. Install watchdog server

  • Run bootstrap-watchdog-server-mycompany build job
    • TODO: Params
  • Download bootstrap-watchdog-server-mycompany from Nexus
    • https://dev.mycompany.ch/nexus/repository/bootstrap/ch/mycompany/bootstrap/bootstrap-watchdog-server-mycompany/$VERSION/bootstrap-watchdog-server-mycompany-$VERSION.zip
  • Setup another server VM
  • Upload bootstrap module to server and install it

Ubuntu:

cd /var/tmp
sudo apt-get install unzip
unzip bootstrap.zip -d bootstrap
cd bootstrap
chmod +x fixture/bootstrap.sh
sudo ./fixture/bootstrap.sh -execute $PWD -flags 'certbot'

Windows:

$global:ProgressPreference = 'SilentlyContinue'
cd C:\tmp
Expand-Archive -LiteralPath bootstrap.zip -DestinationPath bootstrap
cd bootstrap
Set-ExecutionPolicy Unrestricted
.\fixture\bootstrap.ps1 -execute (Get-Location).tostring() -flags 'certbot'

4. Run CD build job

  • Setup another build job for customerone
    • provision.json
  • Include provision step
  • Profit

TODO: Explain the details

5. Run an automated provisioning test

  • Setup vultr account and API
  • Eclipse run configuration for BootstrapTest with correct VM arguments
  • Profit

TODO: Explain the details

Free & Open Source

Bounties

Bounty of CHF 500 is awarded for each meta-client implementations in another language:

  • See Client.xtend/client.json in meta-client for existing implementations
  • Must implement model and protocol classes. See Model.xtend/Protocol.xtend and model.js/protocol.js in meta-core
  • Must implement client interface. See Client.xtend/client.json in meta-core
  • Must implement the same test cases as ClientTest.xtend/client-test.js in meta-engine-test
  • Bounties will be awarded for
    • C# 8
    • Python 3.x
    • Go 1.1x

A bounty of CHF 500 is awarded for reviewing security in meta-engine:

  • Must make recommendations on how to improve existing code to cover different scenarios and attacks
  • Password, OTP and token handling in UserService.xtend
  • Authorization in EngineCore.xtend

Please get in contact before working on a bounty, so we can prevent the unlikely case that multiple people start working on the same thing.

You can receive bounties with:

  • PayPal
  • Wire transfer (must provide IBAN)
  • Gift cards (must be available in Switzerland)
  • TWINT (Switzerland only)

Development / Server

Client

Cheatsheet

Bootstrap

  • *nix installation (Bash with bootstrap.zip in directory /var/tmp)
sudo apt-get install unzip
unzip /var/tmp/bootstrap.zip -d /var/tmp/bootstrap
cd /var/tmp/bootstrap
chmod +x fixture/bootstrap.sh
sudo ./fixture/bootstrap.sh -execute $PWD -start "{moduleDir}/definitions/my-module.json" -flags "clearLogging" -setowner dev
  • Windows installation (PowerShell as Administrator with bootstrap.zip in directory C:/tmp)
$global:ProgressPreference = 'SilentlyContinue'
Expand-Archive -LiteralPath C:\tmp\bootstrap.zip -DestinationPath C:\tmp\bootstrap
cd C:\tmp\bootstrap
Set-ExecutionPolicy Unrestricted
.\fixture\bootstrap.ps1 -execute (Get-Location).tostring() -start '{moduleDir}/definitions/my-module.json' -flags 'clearLogging' -setowner dev
  • Misc
    • If a bootstrap definition is placed in the default directory {moduleDir}/definitions, the full path can be omitted, eg. -start my-module.json instead of -start "{moduleDir}/definitions/my-module.json"

device.js

  • Simulate NFC tag in browser dev tools
window.postMessage(JSON.stringify(device.factory.deviceReadNfcTagEvent({writable: false, id: "1"})), "*");
  • Simulatre NFC tag in WebDriverTest
driver.windowPostDeviceWebViewEvent(new DeviceReadNfcTagEvent(false, "1", null))

Logging

  • Java/Xtend tests in platform
    • See framework/util/util-macro-test/src/main/resources/logback-test.xml
  • Java/Xtend in general
    • See logback.xml or logback-test.xml in src/main/resources
  • Java/Xtend with meta-engine
    • In engine.json
"logging": {
    "loggers": {
        "ch.rswk.meta.engine": "DEBUG"
    }
}
  • JavaScript SPA
    • In config.js with debug: true
  • JavaScript QUnit tests
Logger.useDefaults({
    defaultLevel: Logger.DEBUG
});
  • JavaScript client with meta-admin
    • From tab Clients, copy CID
    • On tab Notifications, send a ClientNotification to the copied CID
      • Channel egn
      • Action INFO
      • Data {"logLevel": 'DEBUG'}

Lucene

  • Inspect index with Luke
    • *nix /usr/local/platform/tool/luke/luke.sh
    • Windows C:\tool\lucene\luke\luke.bat
  • Re-index existing index, eg. after new Lucene version
    • Use clone task from EngineUpdater
  • Re-index existing index, eg. after changes to meta model
    • Implement a EngineUpdaterEntityHandler to migrate existing entities with the clone task
    • See EngineUpdaterSchemaTest.xtend
  • Test with existing index
    • Get copy of index you would like to run locally
      • Manually create a 7z file from the work directory
      • Or backup and download index with meta-admin > File
    • Upload in local meta-admin > File
      • Tags backup,mttluceneservicemeta (for MetaService)
    • Use restore on uploaded file
  • Alternative way to test with existing index
    • Extract/copy index to a local directory, eg. C:/tmp/index/work
    • In MetaServiceTest.xtend, assembleMetaServiceTestConfig, pass directory to assembleLuceneServiceTestConfig, eg. C:/tmp/index

JRE packaging

  • Download JDK
  • Generate list of all .jmod files in jmod directory, eg. with ls and a text editor
  • Generate JRE
    • Start shell in bin directory
    • Replace XXX below with comma separated list of all jmod files (without the .jmod endings)
    • *nix ./jlink --module-path ../jmods --add-modules "XXX" --output /home/dev/java13
    • Windows .\jlink.exe --module-path ..\ --add-modules "XXX" --output C:\data\java13
  • Create 7z from content of java13 directory
    • *nix java13-nix.7z
    • Windows java13-win.7z

Maven

  • Only build current module and not full reactor mvn -N
  • Skip tests mvn -DskipTests
  • Use parallel test feature mvn -T 1C (may break tests)
  • Only build certain modules mvn -pl meta-core,meta-client
  • Only build certain modules and all their dependencies mvn -pl meta-engine -am

meta-admin

  • Run alongside another web module for testing
    • Build meta-admin or download latest release ZIP from Nexus
    • Extract to a folder inside the web module, eg. to myapp-web/target/admin
    • Adapt config.js
      • workerPath: "js/"
    • Run WebDriverTest of the application and go to https://localhost.platform.rswk.ch:9090/myapp-web/target/admin

URLs

  • Tests with StaticResourceServer/PlatformResourceServer
https://localhost.platform.rswk.ch:8088/web-core-test/src/main/resources/qunit.html
  • Tests with meta-engine/HttpService
https://localhost.platform.rswk.ch:9090/meta-admin/src/main/resources/

Web

  • Development/testing
    • In WebDriverTest, at the end of @BeforeEach def void setupRuntime(), insert breakpoint or add System.in.read to halt the test execution. The server is now ready and you can open up the web module, see URLs above
  • Serviceworker
    • See example in meta-admin
      • worker-init.js
      • worker.js
    • Force serviceworker reload
      • In config.js change version
      • In worker-init.js change comment // Version: ${project.version}
  • Force client to reload
    • In meta-admin > Notification send ClientNotification to CID of client
      • Data {"reload": true}

Certificates/keystores

  • Java keystore with self signed
keytool -genkey -alias server -keystore keystore.pfx -storetype PKCS12 -keyalg RSA -validity 720
  • Remove certificate from Java keystore
keytool -delete -alias server -keystore keystore.pfx
  • Install OpenSSL, eg. together with HTTPD (platform/bootstrap/bootstrap-common/src/main/resources/provi/modules/httpd-2.4.json)
  • Configuration
set RANDFILE=C:\data\openssl.rnd
set OPENSSL_CONF=C:\tool\httpd\conf\openssl.cnf
  • Convert PEM certificate and private key to PFX Java keystore
openssl pkcs12 -export -in server.crt -inkey server.key -out server.pfx -name server -password pass:123
  • Self signed CA, server certificate with SAN
    1. Adapt configuration in openssl.cnf
    2. Generate certificate
[ req ]
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = myapp.com

[ v3_ca ]
authorityKeyIdentifier = keyid,issuer
basicConstraints = critical,CA:true,pathlen:3
keyUsage = critical, cRLSign, keyCertSign
nsCertType = sslCA, emailCA
subjectKeyIdentifier = hash
openssl genrsa -out ca.key 2048
openssl req -new -x509 -extensions v3_ca -days 720 -key ca.key -sha256 -out ca.crt
openssl genrsa -out server.key 2048
openssl req -extensions v3_req -sha256 -new -key server.key -out server.csr
openssl x509 -req -extensions v3_req -days 720 -sha256 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -extfile C:\tool\httpd\conf\openssl.cnf

VPN

To generate CA, server and a client cert (multiple client certs can be generated) with OpenVPN on Windows:

  • Download OpenVPN
  • Select EasyRSA Management Scripts in installer
  • Open command line as Administrator in easy-rsa directory
    • init-config.bat
      • Open vars.bat and copy adapted values from below
    • vars.bat
    • clean-all.bat
    • build-ca.bat
      • Common Name and Name: myca
    • build-key-server.bat myserver
    • build-key.bat myclient
      • Common Name and Name: myclient
    • build-dh.bat
  • Find the generated files in easy-rsa/keys folder
Rem vars.bat example
set KEY_COUNTRY=CH
set KEY_PROVINCE=ZH
set KEY_CITY=Zurich
set KEY_ORG=My Company
set KEY_EMAIL=ict@mycompany.ch
set KEY_CN=myserver.mycompany.ch
set KEY_NAME=myserver.mycompany.ch
set KEY_OU=ICT
set PKCS11_MODULE_PATH=changeme
set PKCS11_PIN=1234

Setup server and clients:

  • pfSense VPN server
    • System > Cert Manager
      • CAs > Add
        • Method > Import existing
        • ca.crt
      • Certificates > Add
        • Method > Import existing
        • myserver.crt
        • myserver.key
    • VPN > OpenVPN > Servers > Add
      • Peer Certificate Authority > myca
      • Server certificate > myserver
      • Encryption algorith > AES-256-GCM
      • Enable NCP > Uncheck
      • Auth digest algorithm > SHA512
      • Compression > LZ4 Compression v2
  • pfSense VPN client
    • System > Cert Manager
      • CAs > Add
        • Method > Import existing
        • ca.crt
      • Certificates > Add
        • Method > Import existing
        • myclient.crt
        • myclient.key
    • VPN > OpenVPN > Clients > Add
      • Server host or address > vpn.mycompany.ch
      • Use Automatically generate a TLS Key (and paste it into the VPN server config) if you don’t have one or paste the one already generated on the VPN server
      • Peer Certificate Authority > myca
      • Client certificate > myclient
      • Encryption algorith > AES-256-GCM
      • Enable NCP > Uncheck
      • Auth digest algorithm > SHA512
      • Compression > LZ4 Compression v2
  • OpenVPN server
    • Copy sample-config/server.ovpn to config/server.ovpn
    • Copy the generated files required below to config folder
    • Set/overwrite
      • ca ca.crt
      • cert myserver.crt
      • key myserver.key
      • dh dh2048.pem
      • tls-auth ta.key > Copy generated key
      • cipher AES-256-GCM
      • auth SHA512
      • compress lz4-v2
  • OpenVPN client
auth SHA512
ca [inline]
cert [inline]
cipher AES-256-GCM
client
compress lz4-v2
dev tun
keepalive 10 120
key [inline]
persist-key
persist-tun
port 1194
proto udp
remote vpn.mycompany.ch 1194
remote-cert-tls server
resolv-retry infinite
tls-auth [inline] 1
verb 1
<ca>
INSERT CONTENTS OF ca.crt HERE
</ca>
<cert>
INSERT CONTENTS OF myclient.crt HERE
</cert>
<key>
INSERT CONTENTS OF myclient.key HERE
</key>
<tls-auth>
INSERT CONTENTS OF ta.key (or from pfSense) HERE
</tls-auth>

Links

TODO