Skip to content

Instantly share code, notes, and snippets.

@kjellwinblad
Last active December 21, 2022 09:55
Show Gist options
  • Select an option

  • Save kjellwinblad/c10ce852b9520093a1e1960440e661bf to your computer and use it in GitHub Desktop.

Select an option

Save kjellwinblad/c10ce852b9520093a1e1960440e661bf to your computer and use it in GitHub Desktop.
Description of how to prepare EMQX for hot upgrading

This page describes how to create a new minor release of EMQX Enterprise Edition and test that hot upgrading from the previous minor release to the current one works. The steps to do the same for EMQX Community Edition should be very similar.

  1. Identify which internal apps have changed after the previous minor release

  2. Bump the version number in the APP_NAME.app.src files for the apps that have changed. For example, if the emqx_rule_actions has changed, then bump the version number under the vsn property in lib-ee/emqx_rule_actions/src/emqx_rule_actions.app.src. If the previous version number was "4.4.5" then the new one should be "4.4.6".

  3. Bump the version number for the main EMQX app. This means bumping the version number under the vsn property in the file src/emqx.app.src. If the previous version number was "4.4.5" then the new one should be "4.4.6".

  4. Bump the version numbers in include/emqx_release.hrl. The version numbers in include/emqx_release.hrl should be the same as in src/emqx.app.src.

  5. Add entries for upgrading from the previous minor release to the new release and for downgrading from the new minor release to the previous minor release in src/emqx.appup.src. For example, if the previous release is "4.4.5" and the new release is "4.4.6", then add the following entry to both the list with upgrade instructions and the list with downgrade instructions (i.e., the second and third elements in the tuple which first element is VSN) {"4.4.5",[]}. We leave the list of instructions empty as this list will be populated automatically in the next step.

  6. If you are using a Linux distribution that EMQX doesn't provide specific packages for (e.g., Arch Linux), you need to change the environment variable SYSTEM to fake a system that EMQX has packages for. This is needed since the scripts that are executed in later steps might need to download packages for previous EMQX releases. On most systems, you can set the value of the SYSTEM variable by executing the following in the shell that you are using export SYSTEM=ubuntu20.04.

  7. Run the command ./scripts/update-appup.sh emqx-ee to automatically update the appup files based on changes since the previous release.

  8. Inspect the changes that where done by the previous command (e.g., by running git difftool). As the printout from the ./scripts/update-appup.sh emqx-ee command tells you the automatically added update and downgrade instructions are not perfect. The script cannot detect the correct order for the instructions and cannot detect changes in NIFs.

  9. You are now ready to create a package for the new release by running make emqx-ee-zip. If everything went well, you should have a file named something similar to _packages/emqx-ee/emqx-ee-4.4.6-14adfd82-otp24.1.5-3-ubuntu20.04-amd64.zip.

  10. We should now test that upgrading from the previous release to the newly created release works. Start this process by checking out the code for the previous release in a new directory.

  11. Compile the previous release by running make in its directory

  12. Copy the package for the new release to the releases directory of the previous release (for example, cp emqx_new_rel/_packages/emqx-ee/emqx-ee-4.4.6-14adfd82-otp24.1.5-3-ubuntu20.04-amd64.zip emqx_prev_rel/emqx-enterprise/_build/emqx-ee/rel/emqx/releases/)

  13. Go to the install directory for the previous release (e.g., cd emqx_prev_rel/emqx-enterprise/_build/emqx-ee/rel/emqx)

  14. Start the previous release and test that it works as it should (e.g., ./bin/emqx start)

  15. Perform the upgrade by running ./bin/emqx upgrade 4.4.6-14adfd82. Of course 4.4.6-14adfd82 has to be replaced by the version in your package name. The command should print something similar to the following if everything went well:

    unzip ["releases/emqx-ee-4.4.6-14adfd82-otp24.1.5-3-ubuntu20.04-amd64.zip"]
    Release 4.4.6-14adfd82 not found, attempting to unpack releases/emqx-4.4.6-14adfd82.tar.gz
    Unpacked successfully: "4.4.6-14adfd82".
    Installed Release: 4.4.6-14adfd82.
    Made release permanent: "4.4.6-14adfd82"
    
  16. Now, you can test to see that everything still works as it should after the upgrade

  17. If you want to downgrade so you get to the previous release again, you can run ./bin/emqx upgrade 4.4.5, where 4.4.5 is the nane of the previous release.

@zmstone
Copy link

zmstone commented Jul 8, 2022

Hi @kjellwinblad
Thank you for writing it down.
Could you send it to emqx.git as a PR, maybe put it as docs/relup-development.md

@kjellwinblad
Copy link
Author

@zmstone just saw your message. Yes, will do that.

@kjellwinblad
Copy link
Author

@zmstone emqx.git master branch does not have a docs/ folder. Have that folder moved somewhere else?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment