serverless/event-gateway

Name: event-gateway

Owner: Serverless

Description: React to any event with serverless functions across clouds -

Created: 2017-05-05 15:33:42.0

Updated: 2018-05-24 13:59:31.0

Pushed: 2018-05-24 13:59:36.0

Homepage: https://www.serverless.com/event-gateway

Size: 899

Language: Go

GitHub Committers

UserMost Recent Commit# Commits

Other Committers

UserEmailMost Recent Commit# Commits

README

Event Gateway - React to any event with FaaS function across clouds

We're Hiring! ? Website ? Slack ? Newsletter ? Forum ? Meetups ? Twitter

The Event Gateway combines both API Gateway and Pub/Sub functionality into a single event-driven experience. It's dataflow for event-driven, serverless architectures. It routes Events (data) to Functions (serverless compute). Everything it cares about is an event! Even calling a function. It makes it easy to share events across different systems, teams and organizations!

Use the Event Gateway right now, by running the Event Gateway Getting Started Application with the Serverless Framework.

Features:

The Event Gateway is a L7 proxy and realtime dataflow engine, intended for use with Functions-as-a-Service on AWS, Azure, Google & IBM.

The project is under heavy development. The APIs will continue to change until we release a 1.0.0 version. It's not yet ready for production applications.

Build Status

Event Gateway - Build event-driven integrations with lambda, cloud functions, kubernetes

Contents
  1. Quick Start
  2. Running the Event Gateway
  3. Motivation
  4. Components
    1. Function Discovery
    2. Subscriptions
    3. Spaces
  5. SDKs
  6. Versioning
  7. FAQ
  8. Background
  9. Community
Reference
  1. API
  2. Architecture
  3. Clustering
  4. System Events and Plugin System
  5. Prometheus Metrics
  6. Reliability Guarantees
Quick Start
Getting Started

Looking for an example to get started? The easiest way to use the Event Gateway is with the serverless-event-gateway-plugin with the Serverless Framework. Check out the Getting Started Example to deploy your first service to the Event Gateway.

Running the Event Gateway
Hosted version

If you don't want to run the Event Gateway yourself, you can use the hosted version provided by the Serverless team. Sign up here!

via Docker

There is a official Docker image.

er run -p 4000:4000 -p 4001:4001 serverless/event-gateway -dev
Binary

On macOS or Linux run the following to download the binary:

 -sfL https://raw.githubusercontent.com/serverless/event-gateway/master/install.sh | sh

On Windows download binary.

Then run the binary in development mode with:

ent-gateway -dev

If you want more detailed information on running and developing with the Event Gateway, please check Running Locally and Developing guides.

Motivation
Components
Function Discovery

Discover and call serverless functions from anything that can reach the Event Gateway. Function Discovery supports the following function types:

Function Discovery stores information about functions allowing the Event Gateway to call them as a reaction to received event.

Example: Register An AWS Lambda Function

curl example

 --request POST \
url http://localhost:4001/v1/spaces/default/functions \
header 'content-type: application/json' \
data '{
"functionId": "hello",
"type": "awslambda",
"provider":{
  "arn": "arn:aws:lambda:us-east-1:377024778620:function:bluegreen-dev-hello",
  "region": "us-east-1"
}

Node.js SDK example

t eventGateway = new EventGateway({ url: 'http://localhost' })
tGateway.registerFunction({
nctionId: 'sendEmail',
pe: 'awslambda',
ovider: {
arn: 'xxx',
region: 'us-west-2'


Subscriptions

Lightweight pub/sub system. Allows functions to asynchronously receive custom events. Instead of rewriting your functions every time you want to send data to another place, this can be handled entirely in configuration using the Event Gateway. This completely decouples functions from one another, reducing communication costs across teams, eliminates effort spent redeploying functions, and allows you to easily share events across functions, HTTP services, even different cloud providers. Functions may be registered as subscribers to a custom event. When an event occurs, all subscribers are called asynchronously with the event as its argument.

Creating a subscription requires providing ID of registered function, an event type and a path (/ by default). The path property indicated URL path which Events API will be listening on.

Example: Subscribe to an Event

curl example

 --request POST \
url http://localhost:4001/v1/spaces/default/subscriptions \
header 'content-type: application/json' \
data '{
"type": "async",
"eventType": "user.created",
"functionId": "sendEmail",
"path": "/myteam"

Node.js SDK example

t eventGateway = new EventGateway({ url: 'http://localhost' })
tGateway.subscribe({
pe: 'async',
entType: 'user.created',
nctionId: 'sendEmail',
th: '/myteam'

sendEmail function will be invoked for every user.created event to <Events API>/myteam endpoint.

Example: Emit an Event

curl example

 --request POST \
url http://localhost:4000/ \
header 'content-type: application/json' \
header 'event: user.created' \
data '{ "name": "Max" }'

Node.js SDK example

t eventGateway = new EventGateway({ url: 'http://localhost' })
tGateway.emit({
ent: 'user.created',
ta: { name: 'Max' }

Sync subscriptions via HTTP event

Custom event subscriptions are asynchronous. There is a special http event type for creating synchronous subscriptions. http event is an HTTP request received to specified path and for specified HTTP method. There can be only one http subscription for the same method and path pair.

Example: Subscribe to an “http” Event

curl example

 --request POST \
url http://localhost:4001/v1/spaces/default/subscriptions \
header 'content-type: application/json' \
data '{
"type": "sync",
"eventType": "http.request",
"functionId": "listUsers",
"method": "GET",
"path": "/users"

Node.js SDK example

t eventGateway = new EventGateway({ url: 'http://localhost' })
tGateway.subscribe({
pe: 'sync',
entType: 'http.request',
nctionId: 'listUsers',
thod: 'GET',
th: '/users'

listUsers function will be invoked for every HTTP GET request to <Events API>/users endpoint.

Spaces

One additional concept in the Event Gateway are Spaces. Spaces provide isolation between resources. Space is a coarse-grained sandbox in which entities (Functions and Subscriptions) can interact freely. All actions are possible within a space: publishing, subscribing and invoking.

Space is not about access control/authentication/authorization. It's only about isolation. It doesn't enforce any specific subscription path.

This is how Spaces fit different needs depending on use-case:

Technically speaking Space is a mandatory field (“default” by default) on Function or Subscription object that user has to provide during function registration or subscription creation. Space is a first class concept in Config API. Config API can register function in specific space or list all functions or subscriptions from a space.

SDKs
Versioning

This project uses Semantic Versioning 2.0.0. We are in initial development phase right now (v0.X.Y). The public APIs should not be considered stable. Every breaking change will be listed in the release changelog.

FAQ
What The Event Gateway is NOT
Event Gateway vs FaaS Providers

The Event Gateway is NOT a FaaS platform. It integrates with existing FaaS providers (AWS Lambda, Google Cloud Functions, Azure Functions, OpenWhisk Actions). The Event Gateway enables building large serverless architectures in a unified way across different providers.

Background

SOA came along with a new set of challenges. In monolithic architectures, it was simple to call a built-in library or rarely-changing external service. In SOA it involves much more network communication which is not reliable. The main problems to solve include:

  1. Where is the service deployed? How many instances are there? Which instance is the closest to me? (service discovery)
  2. Requests to the service should be balanced between all service instances (load balancing)
  3. If a remote service call failed I want to retry it (retries)
  4. If the service instance failed I want to stop sending requests there (circuit breaking)
  5. Services are written in multiple languages, I want to communicate between them using the best language for the particular task (sidecar)
  6. Calling remote service should not require setting up new connection every time as it increases request time (persistent connections)

The following systems are solutions those problems:

The main goal of those tools is to manage the inconveniences of network communication.

Microservices Challenges & FaaS

The greatest benefit of serverless/FaaS is that it solves almost all of above problems:

  1. service discovery: I don't care! I have a function name, that's all I need.
  2. load balancing: I don't care! I know that there will be a function to handle my request (blue/green deployments still an issue though)
  3. retries: It's highly unusual that my request will not proceed as function instances are ephemeral and failing function is immediately replaced with a new instance. If it happens I can easily send another request. In case of failure, it's easy to understand what is the cause.
  4. circuit breaking: Functions are ephemeral and auto-scaled, low possibility of flooding/DoS & cascading failures.
  5. sidecar: calling function is as simple as calling method from cloud provider SDK.
  6. in FaaS setting up persistent connection between two functions defeats the purpose as functions instances are ephemeral.

Tools like Envoy/Linkerd solve different domain of technical problems that doesn't occur in serverless space. They have a lot of features that are unnecessary in the context of serverless computing.

Service Discovery in FaaS = Function Discovery

Service discovery problems may be relevant to serverless architectures, especially when we have a multi-cloud setup or we want to call a serverless function from a legacy system (microservices, etc…). There is a need for some proxy that will know where the function is actually deployed and have retry logic built-in. Mapping from function name to serverless function calling metadata is a different problem from tracking the availability of a changing number of service instances. That's why there is a room for new tools that solves function discovery problem rather than the service discovery problem. Those problems are fundamentally different.


This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.