Eventuate Local: Event Sourcing and CQRS with Spring Boot, Apache Kafka and MySQL

Eventuate™  is a platform for developing transactional business applications that use the microservice architecture. Eventuate provides an event-driven programming model for microservices that is based on event sourcing and CQRS.

The benefits of Eventuate include:

  • Easy implementation of eventually consistent business transactions that span multiple microservices
  • Automatic publishing of events whenever data changes
  • Faster and more scalable querying by using CQRS materialized views
  • Reliable auditing for all updates
  • Built-in support for temporal queries

Eventuate™ Local is the open-source version of Eventuate™. It has the same client-framework API as the SaaS version but a different architecture. It uses a MySQL database to persist events, which guarantees that an application can consistently read its own writes. Eventuate Local tails the MySQL transaction log and publishes events to Apache Kafka, which enables applications to benefit from the Apache Kafka ecosystem including Kafka Streams, etc.

This diagram shows the architecture:

Eventuate Local currently only supports Spring Boot applications but we plan to add support for other frameworks and languages over time.

Learn more about Eventuate Local

To find out more about Eventuate Local:

JavaOne 2016: Handling Eventual Consistency in JVM Microservices with Event Sourcing

Last week at JavaOne 2016, Chris Richardson, founder of Eventuate, Inc, and Kenny Bastani, developer advocate at Pivotal, gave a talk on using Event Sourcing to maintain data consistency in a microservices architecture.

Example code

Here is the code for the sample Spring Boot application that Kenny developed for the talk. What is especially exciting is that the microservices demo is built using Eventuate!

Slides

Here are the slides:

Learn more about Eventuate

To find out more about Eventuate:

 

The new Eventuate Java Client

We are super excited to announce that we have started migrating the example applications to the new Eventuate Java client. The highlights of this new client include:

  • Open-source with Javadoc (still work in progress) and source jars
  • Written in Java 8 instead of Scala with a Java wrapper
  • Fully reactive
  • Better modularity, which will make it easier to support more than just Spring applications

So far we have migrated the Money Transfer and the Customer and Orders examples.

For more information, please see the revised getting started guide.

Developing microservices with #DDD aggregates (SpringOne platform, #s1p)

Last week at Spring One Platform, our founder Chris Richardson gave a talk on developing microservices with Domain-Driven Design aggregates.

 

Slides

Here are the slides:

Example code

Here is the code for the Orders and Customers example.

Learn more about Eventuate

To find out more about Eventuate:

Deploying Spring Boot microservices using Docker 1.12 orchestration – part 1

Docker 1.12 was announced earlier this week at Dockercon. Built-in orchestration was one of the particularly interesting new features. This blog post describes how to deploy one of the Eventuate example applications using Docker orchestration.

Install Docker 1.12 on AWS

The first step was to launch three EC2 instances (a master and two worker nodes) and install Docker 1.12.0. We used an Ubuntu  15.10 AMI and ran this script to install Docker 1.12-rc2 on each instance.

You must also had to configure the security group to allow the nodes to communicate as described in the orchestration tutorial:

  • TCP port 2377 for cluster management communications
  • TCP and UDP port 7946 for communication among nodes
  • TCP and UDP port 4789 for overlay network traffic

You must also allow TCP traffic on port 8080 from your local machine in order to be able to access the running application.

Setting up the Swarm

The next step is to set up the Docker swarm, which is a cluster of Docker engines. To do that run docker swarm init on your master node (pick one). Then, run docker swarm join <masterIp>:2377 on each worker node (the other ones). You can verify that the swarm is set up by running the command docker node ls on the master. This command lists the nodes that comprise the swarm.

Build the application

Once the docker swarm is setup you need to build the Kanban sample application on the master node:

# Install Java: 

apt-get install -y openjdk-8-jdk

# Clone the example:  

git clone https://github.com/eventuate-examples/es-kanban-board.git

# Build it: 

cd es-kanban-board/java-server
./gradlew assemble

# Build the docker images:

./docker/build-images.sh

Note: in order to run this application you need to get for credentials for Eventuate event store.

Create the MongoDB service

This application uses MongoDB and so lets create a MongoDB service.

First, we will create a Docker overlay network that the microservices will use to communicate with MongoDB.

docker network create -d overlay kanbannet

Next, we will create the MongoDB service:

docker service create \
  --name mongo  \
  --network kanbannet \
  --replicas 1 \
  -p 27017:27017/tcp \
  mongo:3.0.4

The service runs the mongo 3.0.4 image. Docker orchestration ensures that one instance of Mongo will be running at all times on a node in the swarm.  The -p parameter says that the service is accessible on port 27017.  

A MongoDB client running on an EC2 instance – one of the swarm nodes or elsewhere – can connect to port 27017 on any of the nodes, i.e. masterOrWorkerNodeIpAddress:27017. The Docker routing mesh  routes traffic to the MongoDB container. A swarm service  uses the service name as the DNS name to access MongoDB, i.e. mongo:27017.

Deploy the service

The next step is to deploy the microservices. Rather than deploy the individual services lets first deploy the monolithic version of the application in order to learn how one service (the Java application) connects to another service (MongoDB).

Here is the command to create that service:

docker service create \
  --name standalone \
  --network kanbannet \
  -e EVENTUATE_API_KEY_ID \
  -e EVENTUATE_API_KEY_SECRET \
  -e SPRING_DATA_MONGODB_URI=mongodb://mongo:27017/kanban \
  -p 8080:8080/tcp \
  eventuate_kanban_standalone_service

This service runs the eventuate_kanban_standalone_service image that was built earlier. The -e options specify the eventuate credentials that you received when you signed up and the connection URL for mongodb. Note that the URL uses the mongo hostname, which is the name of the MongoDB service, and is resolved using Docker built-in DNS server.

The service is accessible via port 8080 on every node. Provided that the security group is configured to allow port 8080 traffic  you should be able to access the application from your desktop/laptop using the URL http://<ec2-instance-hostname&gt;:8080.

You can examine the materialized MongoDB views using the MongoDB CLI. You can run that using the following command:

docker run --rm -i -t --net=host mongo:3.0.4 /usr/bin/mongo \
   --host <masterOrWorkerNodeIpAddress>

Note the user of the –net=host option. I discovered that was required in order for the MongoDB client to connect to the server.

What is next

In a later blog post we’ll describe how we deployed the individual microservices

Learn more about Eventuate

To find out more about Eventuate:

 

 

 

 

 

 

Successful software development = organization + process + architecture #gluecon

Successful software development requires the right organizational structure, development processes, and software architecture.

SuccessfulDevelopment.001

Fred Brooks in his book the Mythical Man Month describes how the communication overhead for a team of size n is n(n – 1)/2. Consequently, a software development organization should either be a small team, or  a collection of small, autonomous teams. Amazon, for example,  famously organize around two pizza teams. Pizzas are an ambiguous unit of measurement but probably each team should be 6-10 people.

Despite rumors of the death of agile,  it almost always makes sense to use an agile development process.  Teams should do continuous delivery or, ideally, continuous deployment. Small, agile and autonomous teams can move fast (without breaking things) and keep up with the needs of the business. But what about the software architecture?

For small, simple applications the monolithic architecture often the best choice.  Development is simple, testing is easier and the application is easier to deploy and manage. However, successful application have a habit of growing and your monolithic application will become  large and complex. It will become extremely difficult to develop and deploy in an agile fashion. Teams are no longer autonomous. Delivering software will require lengthy merges and excessive amounts of communication and coordination. You will likely end up in monolithic hell.

Once your application becomes large you will need to adopt the microservices architecture. You functionally decompose what would otherwise be a monolith into a set of small applications or services. Each team owns one or more services. Each service has its own private data in order to ensure loose coupling. This architecture enables the teams to be agile and autonomous.

To learn more about microservices

Take a look at our founder, Chris Richardson’s recent talk at Gluecon on a microservices pattern language.

Learn more about Eventuate

To find out more about Eventuate:

Microservices and Redis – a keynote at #redisconf

Yesterday, Chris Richardson, the founder of Eventuate, gave a keynote about microservices and Redis at RedisConf 2016. Chris described the microservices architecture, explained why you should use it and discussed where Redis fits in.

Slides

Here are the slides:

Example code

There is also an sample application consists of event-driven microservices built using event sourcing and CQRS that uses Redis as a CQRS view.

Learn more about Eventuate

To find out more about Eventuate:

 

Real-time, collaborative Kanban board example built with event sourcing, microservices, @springboot and @docker

We are super excited about the latest Eventuate sample application. It is a real-time, collaborative Kanban board. The application uses Websockets to deliver events published by the Eventuate event store to each user’s browser. This ensures that every user viewing a Kanban board sees the same consistent view . It is a great example of how event sourcing is an excellent choice for real-time collaborative applications.

The following diagram shows the architecture.

The application has an AngularJS front-end. Each of the microservices as well as the API Gateway are written in Java and use Spring Boot. The system-of-record is the Eventuate event store. MongoDB is used as a CQRS view of boards and tasks. The microservices , the API gateway and MongoDB are deployed using Docker.

Event sourcing is a good choice for this kind of collaborative application. That is because whenever the state of the application – i.e. a board or a task – changes an event is published. Those events can then update the information displayed to the users. In this application the events are transformed into WebSocket messages and delivered each user’s browser.

Learn more about Eventuate

To find out more about Eventuate:

 

The Eventuate Todo List application: #microservices, @SpringBoot, #EventSourcing, #CQRS

It is a challenge to write a simple yet meaningful example application that illustrates how to develop event-driven microservices using the Eventuate platform. That is because the microservices architecture requires you to have at least two services. The Money Transfer application has, for example, three (soon to be at least 4) separate services. Even though it is a great example of how to write event-driven microservices using event sourcing and CQRS it is not the easiest introduction to the topic.

To make the getting started experience easier we have created the Todo List application. This application is a Java and Spring Boot application built using Eventuate™’s Event Sourcing based programming model. The application has a HATEAOS-style REST API for creating, querying, updating and deleting a todo list. The API design is inspired by the Todo backend project. The principle difference is that unlike the original version this API is eventually consistent.

The application has a CQRS-based architecture. Creates, updates and deletes are handled by the command side service, which consists of a Todo Aggregate that is persisted using event sourcing. Queries are handled by the query side service, which maintains a materialized view of the todo list in a SQL database.

There are two versions of the application. The simplest version is a single module Gradle project that builds monolithic version of the application of the API. It is a great way to learn the Eventuate API.

The other version of the application is a multi-module Gradle project, which can be deployed as either a monolith or as two microservices: a command-side service and a query-side service.

Both versions come with a Docker Compose file that can be used to launch the application along with a MySQL database.

To get started take a look at the README.md.

Learn more about Eventuate

To find out more about how you can use the Eventuate platform to build event-driven microservices: