Skip to main content

Migrate production nodes

This guide explains how to migrate Actyx production nodes from ActyxOS (v1).

Before you start the migration​

Prepare your PC​

Make sure that you have the latest version of the Node Manager or CLI installed on a computer with which you can access all nodes via the local network. If you have not interacted with nodes before, please also create user keys.

Prepare your deployment​

For a successful migration, no v1 nodes must emit events during and after the migration. This means that you have to stop all apps before you start migrating any node. After all nodes have been migrated, you can start the v2-compatible apps again.

Procedure​

For the migration, it is important to distinguish two types of nodes:

  • Live nodes: These are nodes that have emitted events and still exist, i.e. they are currently running in your v1 deployment
  • Deleted nodes: These are nodes that have emitted events and do not exist anymore, e.g. because hardware was exchanged or Actyx was deleted and re-installed

Before you start with the migration, make a copy of the data directory of a node and move it onto your PC. You will need it later to check if you migrated all nodes and also to migrate any deleted nodes.

caution

Make sure that the node you are copying the directory from is synchronized with other nodes, i.e. it contains all events from all deleted nodes. You can verify that by checking the offset map of the node you pull the data from.

After you have made the copy, you can proceed to migrating all live nodes.

Migrate live nodes​

In order for your apps to keep working as intended after the migration, every live node must rewrite its own events. This means you have to migrate your deployment node-by-node. Additionally to rewriting its own events, your node takes over the following settings:

  • Display name
  • Swarm key
  • Topic
  • Announce addresses
  • initial Peers

Additionally, the node's identity is also preserved.

If you set up a new discovery helper node (previously called bootstrap node) in the previous step, you need to manually change the initialPeers setting of each node to connect to it. If you want to use the same production node as a discovery helper node than before, you do not need to do anything as the setting is automatically migrated and the address is still the same after migration.

The migration procedure depends on the platform your node runs on–it does not matter in which order you migrate your nodes:

tip

We advise that you first make a list of all nodes that should be migrated and update it along the way. Otherwise it will be difficult for you to assess at the end if you actually have deleted nodes that should be migrated, if you just forgot one or more node(s), or if any node is isolated from the rest of the swarm.

caution

Only the events of the currently configured topic are migrated. If you need to migrate events of different topics, please contact us in our community forum or on our Discord server.

Migrating a Windows node works as follows:

  1. Stop your v1 node
  2. Download and install the latest Actyx 2.x release (only works after 2.3)
  3. Stop your v2 node
  4. Copy your v1 data directory
  5. Start your v2 node and migrate
  6. Check if the migration worked

1. Stop your v1 node

Stop your Actyx v1 node by clicking on the tray icon and exit:

migration-01

2. Download and install the latest Actyx 2.x release

You can find installation instructions here.

3. Stop your v2 node

After installation, Actyx automatically starts. In order to stop it, open the Windows Services Manager, and right click on Actyx to stop it:

migration-02

4. Copy your v1 data directory

  1. Open the file explorer and go to the v1 directory under C:\Users\<username>\AppData\Local\ActyxOS, and copy the actyxos-data directory:

migration-03

  1. Navigate to your v2 directory C:\Program Files\Actyx\Node and paste the actyxos-data directory you copied before

  2. Delete the actyx-data directory

  3. Rename the actyxos-data directory to actyx-data:

5. Start your v2 node and migrate

Go back to the Windows Service Manager and start the Actyx Service:

migration-04

5. Ensure that the migration worked

Open the Windows Event Viewer and filter to see the Actyx logs:

migration-05

If you see this log, the migration was successful:

migration-06

You can now uninstall Actyx v1.

After a successful migration, each node publishes an event containing meta data of the migration.

Want to check if you migrated all nodes?

Even if you are sure that you do not have deleted any nodes, we still advise you to perform the next step as it will make you aware of any nodes that you might have forgotten to migrate. Additionally, it will make sure that all v2 nodes are set up to be managed via their admin channel.

Migrate deleted nodes​

As each node only rewrites its own events, events published by deleted nodes are not rewritten yet. Therefore, after you have migrated all live nodes, you can now check for any deleted nodes and migrate them.

In the following example, the v1 deployment consisted of three nodes. One of these nodes was a previously deleted node which is migrated in the example. The other two nodes have already been migrated to v2.

There are a few additional steps you will see below, but in general the migration program works as follows:

  1. Check which nodes of the v1 swarm have already been migrated
  2. Migrate events of nodes that are not migrated yet
  3. Make sure that the migrated events have been distributed to the new v2 swarm

You can download the program for migrating deleted nodes here:

Download
This program needs to be started on the v1 data directory that you copied onto your PC before you started the migration (in this example `actyxos-data-copy`):
./actyx-v1-migration --working-dir actyxos-data-copy

After starting the program, you are prompted with the following information:

migration-09

It shows you the following things:

  • Connected v2 peers: Number of migrated v2 nodes that your PC is connected to
  • Migrated v1 sources: Number of migration events that are in the v2 swarm compared to the number of nodes that were in your v1 swarm (this information is taken from the working directory of the migration program)

Note that, while being connected to only one v2 node, the program shows that two v2 nodes have already been migrated. This is because the v2 node that we are connected already contains the migration information from the other node. One from itself, and the other one from a node that it was previously connected to but the migration program can currently not connect to (e.g. because the tablet is turned off).

In the table, you can see the source ID of the two nodes that were migrated already, and of the node that was not migrated yet. You can navigate (using the arrow keys) and select (using the space key) the deleted nodes that should be migrated.

caution

It is critical to the migration that you now assess if this overview is correct. The problematic scenario is that the overview shows less nodes than you already migrated; In this case, you need to make sure that all nodes you migrated are online and appear as migrated. Otherwise, you might migrate the events of the same nodes and therefore duplicate them!

After selecting the deleted node that should be migrated, you need to enter the private key of your Actyx user keys. This enables the migration program to connect to all nodes of the v2 swarm to make sure that the migrated events of deleted nodes are distributed.

This is using the nodes' Admin channel. The migration program assumes that the nodes listen on port 4458 on the same interface as their swarm port (4001).

After the migrated events were successfully distributed, you see the following table:

migration-10

In the last step, you get an overview of all nodes that you have migrated from v1 to v2 and a mapping between the source ID and stream ID. You can store this information directly, but is it also always accessible by querying the migration events as described in the next guide.

Here is a short video of the workflow (note that the v1_migration program is placed inside the data directory, therefore it does not need to be explicitly specified):

What to do if you find live v1 nodes after finishing the migration

It can happen that you migrated all deleted nodes and later find out that one of them was actually just e.g. tablet you forgot. In this case, just treat it as if it would have been deleted before the migration, i.e. delete Actyx v1 and data from that node. If you would migrate this node again, you would duplicate its events.

If your apps did not use any externally stored offset maps (for e.g. exporting data into other systems), you are finished now and can start your v2-compatible apps on all nodes. Otherwise, continue with our guide on migrating externally stored offset maps.

Problems migrating?

If you have any problems with migrating your apps, please don't hesitate to contact us in our community forum or on our Discord server. We'll do our best to get back to you with answers as fast as possible.