It proved quite hard to find a way to do HMR in ReasonReact.
Here’s a quick but dirty way of doing it:

Initialize an app using reason-scripts
After having installed the bucklescript platform you can run:
yarn create react-app reasonreact-hmr -- --scripts-version reason-scripts

Replace your src/index.re file with this:

[%%bs.raw {|
  require('./index.css')
  var App = require("./app.js");
  var ReactDOMRe = require("reason-react/lib/js/src/reactDOMRe.js");
  var ReasonReact = require("reason-react/lib/js/src/reasonReact.js");

  function render(Component) {
    ReactDOMRe.renderToElementWithId(
      ReasonReact.element(0, 0, Component.make("Welcome to React and Reason", [])), "root");
  }

  render(App);

  if (module.hot) {
    module.hot.accept(() => {
      const NextApp = require('./app').default;
      render(NextApp);
    });
  }
|}];

So what we’ve done here is replace the rendering code which was in Reason before with raw Javascript code which supports hot reloading right from Reason. (The same JS code you can use in any javascript react project to make it support HMR)

Why Javascript and not Reason? Because I don’t know enough of Reason/Ocaml yet to implement this using that.

What’s actually going on?

I took the js code that is generated from the original index.re and modified it to support HMR.
The require statements you see are the ones generated by reason when you run the project for the first time.