The Metamatic Framework

A Simple Alternative to Redux for App Data Management

Posted by Heikki Kupiainen/Oppikone on 06.07.2018 11:45:15

The Easier Way to Handle Data in Your App

This article is for you if you are familiar with ReactJS framework for frontend development and maybe even have already wondered how you should handle data in your browser-based piece of software. This article is also for you if you know what is Redux and ever wondered if there is a more straightforward way to achieve central data management in a web app.

In my previous article I wrote about the minuscule event-dispatching framework with a promise to greatly simplify frontend application development. In this article I will introduce the missing piece of the puzzle and bring in the next generation implemenation of the framework, the Metamatic Framework and show how to use it to create a predictable state container in your JavaScript App.

I have designed the Metamatic Framework, Dear Coder, to make your life easier. It is a Redux-alternative for centrally managing data in your frontend app. Iaam going to highlight the principles of using Metamatic by walking through a metamatic sample application, "The Metamatic Car App". You will find the complete implementation of the sample app here.

In these examples I'll use ES6 syntax but there should be absolutely no problem to use TypeScript or JavaScript either for Metamatic coding!

So What Is the Metamatic Framework?

The Metamatic framework is a little bit of software code - in deed, a very very very tiny bit of software code. Actually it's not at all so much about code, it's rather a new way or concept of managing the data in your app. It is meant to be used together with JavaScript, TypeScript, EcmaScript variants. You can integrate it into your React app or embed it in Angular framework or anything that speaks JavaScript.

People typically use data handling frameworks such as Redux for coding with these languages. I have used Redux in quite some projects. Redux is a great solution to simplify application data management. But it also can be at times frustrating when you must write too much code to get some very simple things done. I want to offer here a more simplistic solution that essentially offers the same core benefits as Redux but needs far less coding from you to get things done.

Why Use Metamatic?

When you code frontend software that run inside browser (or phone or wherever) and once you advance further than creating some "Hello World" apps, you will notice that the issue with data integrity grows in size. You will have many tabs and sections in your app. You implement data retrieval functions that get snippets of data from the server.

It will all go well until you start facing data integrity issues. The user updated some data (their phone number, whatever) on one form but when they enter another tab or section they will see to their dismay that the data they just updated was not changed accordingly on the other tab.

The reason to these kinds of problems is that there are several copies of the same data in different parts of the application. The obvious solution to this is to place all data into one central part of the application. When the user changes the data in some part of the app, then the corresponding data entry is updated in the central data location - from there the change then automatically radiates to all parts of the app where it is being displayed. There are several ways to implement this kind of central data handling.

Getting Started With Metamatic

When you have created your first React app and you have some kind of form for data manipulation, then it's time to connect that form to the central Metamatic data storage. Let's say that we have created a car app web store and we have created a component for showing car details. In CarDetails component, we want to show details of some available car model. Now, we need that CarDetails component to retrieve the car data.

Okay, now that we have CarDetails component, let's import first the essential functions from Metamatic. Add the import statement to the beginning of your class:

javascript import { connect, dispatch} from 'metamatic'i

And don't forget to install metamatic before using it, of course! With Yarn or Npm:

npm install metamatic

or

yarn install metamatic

That's it!

Connect Your Component to the Metamatic Data Store

Now we have all essential data event methods in our use! The next thing we need is to connect our CarDetails class to the Metamatic global data state, which I call MetaStore. MetaStore is the central data store that is responsible for radiating the data to all parts of the app that are listening, when the data changes. When we want to connect a React component to MetaStore, we must define which events we want our React component to listen to. Let's decide that when the car data is loaded from the server by MetaStore, CarDetails component must be updated. The idea is that the local state of the UI component is synchronized with the central metaStore. I'll show here how to do it. First of all, we must make sure that the car data is available for CarDetails component. Therefore we enable CarDetails component's constructor to shout into the bit space that we need car data:

constructor(props) {
    super(props);
    this.state = {};
    dispatch('REQUEST-CAR-DATA');
}

Quite straightforward, isn't it? We have connected CarDetails component to the central data storage with practically one single line of code! Now let's make sure that also the component's local state is synchronized with MetaStore. We must add a handle function that will update the component's state accordingly when CAR-DATA-CHANGE event is received:

constructor(props) {
    super(props);
    this.state = {};
    connect(this, 'CAR-DATA-CHANGE', (carDetails) => this.setState({carDetails}));
}

Quite painless, isn't it? And CarDetails' render method will then function quite normally, interacting with the component's state:

render() {
    let carDetails = this.state.carDetails;
    return carDetails ? (
        <ul className="list-group">
          <li className="list-group-item"><h3>{carDetails.model}</h3></li>
          <li className="list-group-item">Top Speed: {carDetails.speed}</li>
          <li className="list-group-item">Color: {carDetails.color}</li>
          <li className="list-group-item">Price: {carDetails.price}</li>
        </ul>
    ) : null;
}

Implementing a Metamatic Data Storage, MetaStore

Let's implement the actual Metamatic Data Storage, MetaStore. Essentially, MetaStore just needs the exactly same super simple utility functions from metamatic package as do all other parts of the app as well. But now that we want our MetaStore to actually load the datarom server, let's add an AJAX library to MetaStore. I'll use axios but be free to use any other library as you wish. Let's add the essential imports to the beginning of MetaStore.js file:

import { connect, dispatch } from 'metamatic'
import axios from 'axios';

Then, let's define the actual MetaStore central state container:

const metaStore = {
  carData: null
}

Let's also create the initialization function for MetaStore:

export const initMetaStore = () => {};

Now, it's time to get serious. We want to make sure that MetaStore will load car data when it receives request for car data! We expand initMetaStore function to respond to car data request and do something when any component in the application dispatches a request for car data. Let's make metaStore to listen for REQUEST-CAR-DATA event and dispatch a car CAR-DATA-CHANGE event if the car data is available. If not, MetaStore shall delegate the problem and dispatch 'LOAD-CAR-DATA' event. Then some other part in the app can listen to that event and react in an appropriate way. Since MetaStore is not a React component but rather a distinct item that you don't need to unmount or destroy during the App's lifetime, you can use handle instead of connect to listen for events:

export const initMetaStore = () => {
   handle('REQUEST-CAR-DATA', () => {
      if (metaStore.carData) {
         return dispatch('CAR-DATA-CHANGE', metaStore.carData)   
      }       
      dispatch('LOAD-CAR-DATA');n
    });
};

The example shown above works as a sort of a cache as well. When the car data was already available it won't be loaded from theserver. Rather the handler will dispatch the existing car data into the app-wide bit space. Of course it's a good question whether caching data in the browser is desirable at all in the first place. Some even say that web sockets are a better way for data communication between a browser based app and the server - and they might be even right! But let's address this question later on.

Now we need, of course, someone to listen for 'LOAD-CAR-DATA' event. That listener will need to load the actual data when the event was received. We can implement this event handler wherever we like but It's conventient to do it inside the MetaStore now that it's not all too big at this point. When the file size grows too big, I think, ethen it's the right time to split handlers into separate files. Let's add the car data loading request handler inside initMetaStore function:

export const initMetaStore = () => {
   
   ...
   
   handle('LOAD-CAR-DATA', () => {
      axios.get(`${YOUR_CAR_DATA_URL_HERE}`)
         .then(response => {
             dispatch('LOAD-CAR-DATA-SUCCESS', response.data)
         })
        .catch(error => dispatch("LOAD-CAR-DATA-ERROR", error));
      })
   });
};

Now let's decide what to do when the requested data was loaded from the server. We just get the response data from the server and place it into the central MetaStore. And most kindly, we also notify about car data change so all related components can update themselves with the new fresh data. Note that I am cloning the data with spread operator {...data} to prevent the central data being changed by a hideous direct manipulation somewhere else int the app!

handle('LOAD-CAR-DATA-SUCCESS', (data) => {
   metaStore.carData = data;
   dispatch('CAR-DATA-CHANGE', {...metaStore.carData});
});

After defining these handlers, the MetaStore.js file should appear as follows:

import {dispatch, handle} from 'metamatic'
import axios from 'axios';
import {CAR_DATA_URL} from '../constants';

const metaStore = {
  carDetails: null
};

export const initMetaStore = () => {

  handle('REQUEST-CAR-DATA', () => {
    if (metaStore.carData) {
      return dispatch('CAR-DATA-CHANGE', metaStore.carData)
    }
    dispatch('LOAD-CAR-DATA');
  });

  handle('LOAD-CAR-DATA', () => {
    axios.get(`${CAR_DATA_URL}`)
    .then(response => {
      dispatch('LOAD-CAR-DATA-SUCCESS', response.data)
    }).catch(error => dispatch("LOAD-CAR-DATA-ERROR", error));
  });

  handle('LOAD-CAR-DATA-SUCCESS', (data) => {
    metaStore.carDetails = data;
    dispatch('CAR-DATA-CHANGE', {...metaStore.carDetails});
  });
};

Note that I am here splitting the logic into many sections that are quite loosely coupled to each other. Different parts of the logic live their independent life, being loosely connected only through events. There are many advocates of this kind of event-driven approach. However, I would warn against going too event-driven in the first place. The idea of events is to disconnect parts of the application from each other in the name of modularity. But disconnectedness may actually be a very bad thing. It's easy for a developer to follow the logic by clicking on a method call when using a proper editor. But when you have broken the direct connection between two different methods by joining them together not through direct method invocation but rather via events - with one function firing an event and another listening for it - you have just created an extra layer of complexity. From then on you will always need a bit more detective work to follow the application logic. You must search where the event was defined and where it's being expected. It makes the code more difficult to read. And the software code is always written as much for humans as for the machine. This concern is a major justification for writing very standard code always when possible - avoiding too much innovation - even if it results in somewhat lower results/time spent ratio. But there will always be unique problems that can only be addressed with unique code.

Turning Your Events Into Constants

When dispatching and handling events, you can always use plain strings, as in the previous examples. But this practice will become hazardous over time. If you make a typing mistake when firing events, you may end up puzzled when you are trying to figure out why your handler doesn't appear to react to the event. It may be worth of a debatte to ask whether constants should be defined in a dedicated file or should you rather introduce them as a part of an existing file. My opinion is that the ideal place the event constants is right in the file where the MetaStore resides. When MetaStore grows bigger, it might be worth of considering to separate the constants in a separate file. Until then, the constantized MetaStore file will appear as follows:

import {dispatch, handle} from 'metamatic'
import axios from 'axios';
import {CAR_DATA_URL} from '../constants';

export const REQUEST_CAR_DATA = 'REQUEST_CAR_DATA';
export const CAR_DATA_CHANGE = 'CAR_DATA_CHANGE';
export const LOAD_CAR_DATA = 'CAR_DATA_CHANGE';
export const LOAD_CAR_DATA_SUCCESS = 'LOAD_CAR_DATA_SUCCESS';
export const LOAD_CAR_DATA_ERROR = 'LOAD_CAR_DATA_ERROR';

const metaStore = {
  carDetails: null
};

export const initMetaStore = () => {

  handle(REQUEST_CAR_DATA, () => {
    if (metaStore.carData) {
      return dispatch(CAR_DATA_CHANGE, metaStore.carData)
    }
    dispatch(LOAD_CAR_DATA);
  });

  handle(LOAD_CAR_DATA, () => {
    axios.get(`${CAR_DATA_URL}`)
    .then(response => {
      dispatch(LOAD_CAR_DATA_SUCCESS, response.data)
    }).catch(error => dispatch(LOAD_CAR_DATA_ERROR, error));
  });

  handle('LOAD-CAR-DATA-SUCCESS', (data) => {
    metaStore.carDetails = data;
    dispatch(CAR_DATA_CHANGE, {...metaStore.carDetails});
  });
};

Please check GitHub for changes made to turn MetaStore events into constants here. To check the complete source file of this MetaStore implementation please check here.

Initializing the MetaStore

When you have implemented MetaStore, make sure to actually initialize it by calling initMetaStore function. A good play to do this is at the earliest point of the Application wakeup, for instance in the constructor of App's main component, for example App.js, as many React apps tend to have as their main object:

class App extends React.Component {

  constructor(props) {
    super(props);
    initMetaStore();
    ...
  }
}

Create the Mock Server

When you start creating any serious frontend app, be sure to implement a mock server for serving fake data for your frontend app as soon as possible. There may be some people saying that you shouldn't use mock data at all but rather rely on real data. That may be true at some point of the application development cycle. But that point comes into question rather later on when the design meets the very minimal definition of maturity and there is already some insight into the needed database structure.

But when you are implementing a new solution from scratch, I highly recommend to implement a mock server in the very beginning of the project, until the business department has come up with a clearer vision about how the software should actually function. If you wait for the "final data" before starting to implement the frontend, you will have a grave chicken-and-egg problem. The business won't ever be able to envision what they really want if they don't have the inital app - even with mock data - that they can play with to decide how the app should work. Therefore you must create a mock data server so you can speed up quickly to implement a frontend based on the early wireframes. When the business department can test out how the demo works then the ideas will be pouring out of these experiences like a fountain of wisdom - helping define the required database structure. If you start creating the software upside down, starting from database and server-side implementation, it will surely slow down all development to
snail speed because you don't even know in the first place how the data should be structured to make the optimal app possible. All changes to the frontend will be extremely slow and expensive - of course depending a bit on how heavy the server side design is.

Frontend is the starting point for new innovative software solutions, not the backend. Therefore the backend must be as light as possible in the very formative phase of the new software, so it can adapt fast to even radical UI design changes that will be coming in first with a rapid cycle, then somewhat slower and more moderate when insight and consensus start to form up.

Creating the initial mock server for the evolving app is actually ridiculously easy:

var bodyParser = require('body-parser');
var cors = require('cors');
var express = require('express');
var path = require('path');
var app = express();

app.use(bodyParser.urlencoded({
  extended: true
}));

app.use(bodyParser.json());
app.use(cors());
app.use(express.static('public'));

app.use('/app', express.static('public'))
app.use('/app', express.static(path.join(__dirname, 'public')))

const getCars = () => [
  {
    'id': 1,
    'model': 'Audi',
    'speed': 170,
    'color': 'black',
    'price': 28500,
    'marketShare': []
  },
  {
    'id': 2,
    'model': 'Porsche',
    'speed': 250,
    'color': 'white',
    'price': 61000,
    'marketShare': []
  },
  {
    'id': 3,
    'model': 'Tesla',
    'speed': 280,
    'color': 'green',
    'price': 79000,
    'marketShare': []
  },
  {
    'id': 4,
    'model': 'Ferrari',
    'speed': 320,
    'color': 'red',
    'price': 344000,
    'marketShare': []
  },
  {
    'id': 5,
    'model': 'Lamborghini',
    'speed': 334,
    'color': 'yellow',
    'price': 521000,
    'marketShare': []
  },
  {
    'id': 6,
    'model': 'Bugatti',
    'speed': 420,
    'color': 'blue',
    'price': 1591000,
    'marketShare': []
  }
];

const getCarsMap = () => getCars().reduce((map, car) => {
  map[car.id] = car;
  return map;
}, {});

const getCarById = (id) => getCarsMap()[id];

app.get('/api/cars', function (req, res) {
  const id = req.params.id;
  res.json(getCars());
});

app.get('/api/cars/:id', function (req, res) {
  const id = req.params.id;
  res.json(getCarById(id));
});

module.exports = app;
app.listen(3001);

What we did here is that we implement two endpoints, one for getting many cars and one for retrieving details of one car only. While we want to keep the server simple at this point, the data is hard-coded only so we don't need to integrate the server with a database yet.

Strip Off Complexity With Central Event Handling

When you start creating frontend software with ReactJS and implement new components, you will notice over time how your component hierarchy grows into a tree-like structure. It will become challenging to pass events from leaf components downward towards the root component. For instance, when the user clicks on LoginButton component on LoginForm component, you must implement a click event dispatch function inside LoginButton and then another handler function in its parent LoginForm component and so on. LoginForm component then forwards event further to its parent and so on, until the event reaches the component that can do the actually login validation.

This kind of event "bubbling" through hierarchy creates incredibly much unnecessary complexity in your app because you must remember to implement an ever growing chain of event dispatchers and handlers all the way through the hierarchy starting from the leaf until the very root component. This kind of event bubbling degrades the application over time into some seriously unmaintainable bubble gum, indeed! All changes in the logic will be slow and error prone to implement because of the chain reaction of changes that will always be required even on the slightest update in the application logic. But you can avoid this kind of bubble gum complexity completely if you implement a central location for handling events, the MetaStore! Have a look at the Login component that dispatches Login form data directly to MetaStore when the user clicks on 'Submit' button. This implementation makes the Login component independent from its parent. The parent does not need to pass any event listeners as props to this component, it only needs to know where to place it!

import React from 'react';
import {dispatch} from 'metamatic';
import {SUBMIT_LOGIN} from '../store/MetaStore';

export class Login extends React.Component {

  constructor(props) {
    super(props);
    this.state = {
      email: '',
      password: ''
    };
  }

  updateEmail = (event) => this.setState(...this.state, {email: event.target.value});

  updatePassword = (event) => this.setState(...this.state, {password: event.target.value});

  handleSubmit = (event) => dispatch(SUBMIT_LOGIN, this.state);

  render = () => (
      <form>
        <div className="form-group">
          <label htmlFor="email">Email address:</label>
          <input type="email" className="form-control" id="email" 
        onChange={this.updateEmail}/>
        </div>
        <div className="form-group">
          <label htmlFor="pwd">Password:</label>
          <input type="password" className="form-control" id="pwd" 
        onChange={this.updatePassword}/>
        </div>
        <div className="checkbox">
          <label><input type="checkbox"/> Remember me</label>
        </div>
        <button type="submit" className="btn btn-default" 
        onClick={this.handleSubmit}>Submit</button>
      </form>
  );
}

In the implementation above, all keystrokes made in the input fields will cause this component's local state to change. But when the user clicks on the submit button, the local state will be dispatched inside SUBMIT_LOGIN event. Thus, this solution has its own private local state but is also communicating with the central MetaStore state. Some say that components shouldn't have any local state at all but have all their states stored in a central location instead. I think such view is rather academic and serves no real purpose. Encapsulation of data is a very serious principle and should not be forgotten in the excitement of some new yet-another hype "paradigm".

In my opinion, all properties that are not needed all the time by outsiders can and should be stored in the local state. In this case, other parts of the application will need to know Login form's email and password properties only after the submit button has been clicked. And that's exactly what we are doing in this exercise!

Access Logic, Simple versus RBAC

Now that LoginForm component can fire a SUBMIT_LOGIN event, there must be a handler in the MetaStore to receive that event and perform an AJAX request accordingly to post the authentication request to the server:

  handle(SUBMIT_LOGIN, (credentials) => {
    axios.post(LOGIN_URL, credentials)
    .then(response => {
      metaStore.loggedIn = response.data.loggedIn;
      dispatch(LOGIN_STATE_CHANGE, metaStore.loggedIn)
    }).catch(error => dispatch(LOGIN_FAILURE, error));
  });

This simple authentication implementation will be adequate in the formative period of the Metamatic Car App but remember that when your Car App grows into production-quality business, you will need a more complex authentication and access control scheme - you will need a serious RBAC (role-based access control) and you may need to enable login through existing channels that the users prefer to use, such as Facebook, Google or Twitter login, whatever. They hate creating unique credentials for each service. You will need an entire system not smaller than your car app itself just for managing user accounts; adding new accounts, removing old accounts, sending automated sign-up confirmation emails, re-sending password reset links, maybe setting expiration dates etc. If you start implementing all these administrative features on your own, your team will find themselves in a quagmire burning their budget once again reinventing something that has been implemented many times over before. I think those teams will win the race that don't try to do everything by themselves so they can concentrate on the core business logic. They rather use existing authentication solutions such as Auth0. I will be explaining later how to integrate Auth0 to enable RBAC for your app in just couple of hours, not in weeks or months as you may have seen sometime in the past!

But this being said, we still need to start with a naive solution to implement simplest possible login first - we will need to get something working very fast and then later on gradually improve the solution step by step. Until then, we have to be happy with a naive solution so we can really storm to implement the actual logic!

In the Metamatic Car App there is a main class to hold the visual components. It's called App.js. Depending on whether the user has logged in or not, App component will display either Login or Management component:

import React from 'react';
import ReactDOM from 'react-dom';
import './App.css';

import {AppTitle} from './component/AppTitle.js';
import {Management} from './component/Management.js';
import {Login} from './component/Login.js';

import {connect} from 'metamatic';
import {initMetaStore, LOGIN_STATE_CHANGE, LOGOUT} from './store/MetaStore';

export class App extends React.Component {

  constructor(props) {
    super(props);
    initMetaStore();
    this.state = {};
    connectt(LOGIN_STATE_CHANGE, (loggedIn) => {
        this.setState({loggedIn});
    })
  }

  createViewComponent() {
    if (this.state.loggedIn) {
      return <Management/>;
    } else {
      return <Login/>;
    }
  }

  render() {
    let viewComponent = this.createViewComponent();
    return (
        <div className="container-fluid">
          <AppTitle name="Cars"/>
          {viewComponent}
        </div>
    );
  }
}

ReactDOM.render(
    <App/>,
    document.getElementById('root')
);

In the example above, MetaStore will be firing LOGIN_STATE_CHANGE events depending on what happens. When authentication was succesful, the value for loggedIn will be true and when the user clicks on EXIT button, it will lead MetaStore to dispatching loggedIn as false within LOGIN_STATE_CHANGE event.

Let's have a look at logging out definition inside MetaStore. We just add:

handle(LOGOUT, () => {
  metaStore.loggedIn = false;
  dispatch(LOGIN_STATE_CHANGE, metaStore.loggedIn);
})

When MetaStore receives LOGOUT event from Navigation component it then just updates its loggedIn state to false and dispatches this new state further to anybody that listens LOGIN_STATE_CHANGE event.

Coming Soon...

Either way, I hope these examples highlight the idea of a very easy, very straightforward, over-engineering-free approach to implement robust central data storage without too complex frorks. I can barely wait to continue highlighting The Metamatic Approach: An event-driven, minimalistic, simple way to manage data in your app. I'll be back to you soon with more examples about the topic!