CD2H gitForager

cloudfoundry-samples/ratelimit-service

Name: ratelimit-service

Owner: Cloud Foundry Sample Applications

Description: Simple rate limiting route service

Created: 2016-02-11 15:55:04.0

Updated: 2017-11-13 16:15:16.0

Pushed: 2016-03-22 16:46:38.0

Homepage: null

Size: 2085

Language: Go

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

Rate Limiting Route Service

About

A simple in-memory rate limiting route service for Cloud Foundry.

This rate limiter route service app is a forwarding proxy that will limit the number of requests/second per client IP address. For example, maximum 10 requests per second per client.

If you would like more information about Route Services in Cloud Foundry, please refer to CF dev docs.

NOTE: This is an example only and is not for production use, but enough with the disclaimers…

Prerequisites

A Cloud Foundry and Diego deployment
CF CLI v6.16+
an app deployed and running on Cloud Foundry you want to rate limit

this rate limiter app

t clone https://github.com/cloudfoundry-samples/ratelimit-service.git

The examples below are using bosh-lite
The app you would like rate limit is running at http://myapp.bosh-lite.com and running on Diego runtime.

Install

Deploy Rate Limiter App

 ratelimit-service
 push ratelimiter

The rate limiter proxy app will now be running at: https://ratelimiter.bosh-lite.com.

(Optional) Configure limit of requests per second

To override the default limit (10), you can set the following application env var and restage:

 set-env ratelimiter RATE_LIMIT 1
 env ratelimiter

-Provided:
_LIMIT: 1

 restage ratelimiter

(Optional) Skip SSL Validation

If you set the following environment variable to false, the route service will validate SSL certificates. By default the route service skips SSL validation.

 set-env ratelimiter SKIP_SSL_VALIDATION false
 restage ratelimiter

Create Route Service

The following will create a route service instance using a user-provided service and specifies the route service url (see step above).

 create-user-provided-service ratelimiter-service -r https://ratelimiter.bosh-lite.com
ting user provided service ratelimiter-service in org my-org / space as admin...

Bind Route to Service Instance

The following will create bind the application's route to the route service instance.

 bind-route-service bosh-lite.com ratelimiter-service --hostname myapp
ing route myapp.bosh-lite.com to service instance ratelimiter-service in org my-org / my-space as admin...

Try it out

To test the rate limiting, you will need to exceed the requests / second limit.

requests made that are within the limit will return an HTTP 200
requests made that exceed the limit will return an HTTP 429 - Too Many Requests

In the examples below we will use a rate limit of 10 requests / per second per client IP

Client-side tool

There is a great command line tool (similar to Apache Bench) called boom which allows you to send a number of requests and also throttle the number of concurrent client requests.

To install client side load testing tool:

et github.com/rakyll/boom

Example (not exceeding rate limit)

For example: 100 requests, 10 concurrently and a QPS of 10

om -n 100 -c 10 -q 10 http://myapp.bosh-lite.com
/ 100 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooom! 100.00 %

ary:
tal:        10.1374 secs.
owest:      0.7535 secs.
stest:      0.0233 secs.
erage:      0.1478 secs.
quests/sec: 9.8644
tal Data Received:  2500 bytes.
sponse Size per Request:    25 bytes.

us code distribution:
00] 100 responses

Example (exceeding rate limit)

For example: 100 requests, 12 concurrently and a QPS of 12

In this example, since we will be sending 12 requests / sec, then only roughly 84% of the requests should be successful.

om -n 100 -c 12 -q 12 http://myapp.bosh-lite.com
/ 100 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooom! 100.00 %
ary:
tal:        8.3826 secs.
owest:      0.6653 secs.
stest:      0.0352 secs.
erage:      0.1126 secs.
quests/sec: 11.9295
tal Data Received:  2388 bytes.
sponse Size per Request:    23 bytes.

us code distribution:
00] 86 responses
29] 14 responses

Unbinding Route Service

If you want to turn off rate limiting, the following will unbind the application's route from the route service instance.

nbind-route-service bosh-lite.com ratelimiter-service --hostname myapp

nding may leave apps mapped to route myapp.bosh-lite.com vulnerable; e.g. if service instance ratelimiter-service provides authentication. Do you want to proceed?> yes
nding route myapp.bosh-lite.com from service instance ratelimiter-service in org my-org / space my-space as admin...

Logs

You can watch the logs of the rate limiter app to see when requests come it for given IP addresses

 logs ratelimiter
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.49-0600 [App/0]  OUT request from [10.244.0.25]
6:20.50-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38101 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:127716be-9cd7-484d-634c-d1051993accb response_time:0.013694649 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.51-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38102 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:cf60dc83-bff1-4e53-6ff7-43a52667e2a4 response_time:0.017450561 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.52-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38109 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:272a34cc-dcaa-4c13-5584-be28e25855a2 response_time:0.027553184 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.52-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38107 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:6c379238-d5fc-4fc5-7b27-e4776d830c41 response_time:0.029788514 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.53-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38106 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:2efdd43d-3429-4869-54f6-634cb4bc9854 response_time:0.038401721 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.53-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38103 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:2f059c95-31b9-42ab-4459-1fe0f54dc9fa response_time:0.038841171 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.54-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38108 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:950c9525-490a-4fd1-7b38-5f74514ea085 response_time:0.046893267 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.55-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38104 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:49c5650b-c511-4fe5-7f56-067e404d7b8b response_time:0.059563104 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.55-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38105 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:de96c743-032f-4977-4466-f9241a03cf6a response_time:0.063431108 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.56-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 200 0 25 "-" "Go-http-client/1.1" 10.244.0.21:38111 x_forwarded_for:"192.168.50.1, 10.244.0.21" x_forwarded_proto:"http" vcap_request_id:081cb579-a038-4511-43e9-b8f2074019b9 response_time:0.070817607 app_id:7a1745bc-d7cb-43a3-8201-c6ac5d75e79c
6:20.64-0600 [App/0]  OUT request from [10.244.0.25]
6:20.64-0600 [App/0]  OUT request from [10.244.0.25]
6:20.64-0600 [App/0]  OUT rate limit exceeded for 10.244.0.25
6:20.64-0600 [RTR/0]  OUT ratelimiter.bosh-lite.com - [20/02/2016:20:16:20 +0000] "GET / HTTP/1.1" 429 0 17 "-" "Go-http-client/1.1" 10.244.0.21:38144 x_forwarded_for:

Misc

The rate limit app also has a /stats endpoint that displays the current list of IPs and available requests, which can be useful for debugging or displaying current stats.

For example, if the limit is set to 10 then the “available” number will be a number between 0 and 10, with zero meaning that the (req/sec) rate limit has been exceeded for the client IP.

rl ratelimiter.bosh-lite.com/stats

son


"ip": "10.244.0.25",
"available": 3


"ip": "10.244.0.29",
"available": 0

This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.