Name: localstack
Owner: BBC
Description: ? A fully functional local AWS cloud stack. Develop and test your cloud apps offline!
Created: 2018-05-17 07:58:28.0
Updated: 2018-05-17 07:58:31.0
Pushed: 2018-05-11 19:13:45.0
Homepage: https://localstack.cloud
Size: 1732
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
LocalStack provides an easy-to-use test/mocking framework for developing Cloud applications.
Currently, the focus is primarily on supporting the AWS cloud stack.
0.7.0, the Docker image will be pushed
and kept up to date under the new name localstack/localstack. (This means that you may
have to update your CI configurations.) Please refer to the updated
End-User License Agreement (EULA) for the new versions.
The old Docker image (atlassianlabs/localstack) is still available but will not be maintained
any longer.LocalStack spins up the following core Cloud APIs on your local machine:
Additionally, LocalStack provides a powerful set of tools to interact with the cloud services, including a fully featured KCL Kinesis client with Python binding, simple setup/teardown integration for nosetests, as well as an Environment abstraction that allows to easily switch between local and remote Cloud execution.
LocalStack builds on existing best-of-breed mocking/testing tools, most notably kinesalite/dynalite and moto. While these tools are awesome (!), they lack functionality for certain use cases. LocalStack combines the tools, makes them interoperable, and adds important missing functionality on top of them:
ProvisionedThroughputExceededException which is thrown by Kinesis or DynamoDB if the amount of
read/write throughput is exceeded.@mock_sqs. These client proxies
do not perform an actual REST call, but rather call a local mock service method that lives in the same process as
the test code.makepython (both Python 2.x and 3.x supported)pip (python package manager)npm (node.js package manager)java/javac (Java 8 runtime environment and compiler)mvn (Maven, the build system for Java)The easiest way to install LocalStack is via pip:
install localstack
Once installed, run the infrastructure using the following command:
lstack start
Note: Please do not use sudo or the root user - LocalStack
should be installed and started entirely under a local non-root user.
If you have problems with permissions in MacOS X Sierra,
install with pip install --user localstack
You can also spin up LocalStack in Docker:
lstack start --docker
(Note that on MacOS you may have to run TMPDIR=/private$TMPDIR localstack start --docker if
$TMPDIR contains a symbolic link that cannot be mounted by Docker.)
Or using docker-compose (you need to clone the repository first):
er-compose up
(Note that on MacOS you may have to run TMPDIR=/private$TMPDIR docker-compose up if
$TMPDIR contains a symbolic link that cannot be mounted by Docker.)
You can pass the following environment variables to LocalStack:
SERVICES: Comma-separated list of service names and (optional) ports they should run on.
If no port is specified, a default port is used. Service names basically correspond to the
service names of the AWS CLI
(kinesis, lambda, sqs, etc), although LocalStack only supports a subset of them.
Example value: kinesis,lambda:4569,sqs:4570 to start Kinesis on the default port,
Lambda on port 4569, and SQS on port 4570.
DEFAULT_REGION: AWS region to use when talking to the API (defaults to us-east-1).
HOSTNAME: Name of the host to expose the services internally (defaults to localhost).
Use this to customize the framework-internal communication, e.g., if services are
started in different containers using docker-compose.
HOSTNAME_EXTERNAL: Name of the host to expose the services externally (defaults to localhost).
This host is used, e.g., when returning queue URLs from the SQS service to the client.
USE_SSL: Whether to use https://... URLs with SSL encryption (defaults to false).
KINESIS_ERROR_PROBABILITY: Decimal value between 0.0 (default) and 1.0 to randomly
inject ProvisionedThroughputExceededException errors into Kinesis API responses.
DYNAMODB_ERROR_PROBABILITY: Decimal value between 0.0 (default) and 1.0 to randomly
inject ProvisionedThroughputExceededException errors into DynamoDB API responses.
LAMBDA_EXECUTOR: Method to use for executing Lambda functions. Possible values are:
local: run Lambda functions in a temporary directory on the local machinedocker: run each function invocation in a separate Docker containerdocker-reuse: create one Docker container per function and reuse it across invocationsFor docker and docker-reuse, if LocalStack itself is started inside Docker, then
the docker command needs to be available inside the container (usually requires to run the
container in privileged mode). Default is docker, fallback to local if Docker is not available.
LAMBDA_REMOTE_DOCKER determines whether Lambda code is copied or mounted into containers.
Possible values are:
true (default): your Lambda function definitions will be passed to the container by
copying the zip file (potentially slower). It allows for remote execution, where the host
and the client are not on the same machine.false: your Lambda function definitions will be passed to the container by mounting a
volume (potentially faster). This requires to have the Docker client and the Docker
host on the same machine.DATA_DIR: Local directory for saving persistent data (currently only supported for these services:
Kinesis, DynamoDB, Elasticsearch, S3). Set it to /tmp/localstack/data to enable persistence
(/tmp/localstack is mounted into the Docker container), leave blank to disable
persistence (default).
PORT_WEB_UI: Port for the Web user interface (dashboard). Default is 8080.
<SERVICE>_BACKEND: Custom endpoint URL to use for a specific service, where <SERVICE> is the uppercase
service name (currently works for: APIGATEWAY, CLOUDFORMATION, DYNAMODB, ELASTICSEARCH,
KINESIS, S3, SNS, SQS). This allows to easily integrate third-party services into LocalStack.
FORCE_NONINTERACTIVE: when running with Docker, disables the --interactive and --tty flags. Useful when running headless.
Additionally, the following read-only environment variables are available:
LOCALSTACK_HOSTNAME: Name of the host where LocalStack services are available.
This is needed in order to access the services from within your Lambda functions
(e.g., to store an item to DynamoDB or S3 from Lambda).
The variable LOCALSTACK_HOSTNAME is available for both, local Lambda execution
(LAMBDA_EXECUTOR=local) and execution inside separate Docker containers (LAMBDA_EXECUTOR=docker).You can point your aws CLI to use the local infrastructure, for example:
--endpoint-url=http://localhost:4568 kinesis list-streams
"StreamNames": []
NEW: Check out awslocal, a thin CLI wrapper that runs commands directly against LocalStack (no need to
specify --endpoint-url anymore). Install it via pip install awscli-local, and then use it as follows:
ocal kinesis list-streams
"StreamNames": []
UPDATE: Use the environment variable $LOCALSTACK_HOSTNAME to determine the target host
inside your Lambda function. See Configurations section for more details.
boto3 and use the endpoint_url parameter when creating a connectionIf you want to use LocalStack in your integration tests (e.g., nosetests), simply fire up the infrastructure in your test setup method and then clean up everything in your teardown method:
localstack.services import infra
setup():
infra.start_infra(async=True)
teardown():
infra.stop_infra()
my_app_test():
# here goes your test logic
See the example test file tests/test_integration.py for more details.
In order to use LocalStack with Java, the project ships with a simple JUnit runner. Take a look
at the example JUnit test in ext/java. When you run the test, all dependencies are automatically
downloaded and installed to a temporary directory in your system.
rt cloud.localstack.LocalstackTestRunner;
rt cloud.localstack.TestUtils;
With(LocalstackTestRunner.class)
ic class MyCloudAppTest {
est
blic void testLocalS3API() {
AmazonS3 s3 = TestUtils.getClientS3()
List<Bucket> buckets = s3.listBuckets();
...
Additionally, there is a version of the LocalStack Test Runner which runs in a docker container instead of installing LocalStack on the current machine. The only dependency is to have docker installed locally. The test runner will automatically pull the image and start the container for the duration of the test. The container can be configured by using the @LocalstackDockerProperties annotation.
With(LocalstackDockerTestRunner.class)
alstackDockerProperties(randomizePorts = true, services = { "sqs", "kinesis:77077" })
ic class MyDockerCloudAppTest {
est
blic void testKinesis() {
AmazonKinesis kinesis = DockerTestUtils.getClientKinesis();
ListStreamsResult streams = kinesis.listStreams();
...
The LocalStack JUnit test runner is published as an artifact in Maven Central.
Simply add the following dependency to your pom.xml file:
endency>
<groupId>cloud.localstack</groupId>
<artifactId>localstack-utils</artifactId>
<version>0.1.13</version>
pendency>
If you're using AWS Java libraries with Kinesis, please, refer to CBOR protocol issues with the Java SDK guide how to disable CBOR protocol which is not supported by kinesalite.
Accessing local S3 from Java: To avoid domain name resolution issues, you need to enable path style access on your client:
etS3ClientOptions(S3ClientOptions.builder().setPathStyleAccess(true).build());
here is also an option to do this if you're using any of the client builder classes:
onS3ClientBuilder builder = AmazonS3ClientBuilder.standard();
der.withPathStyleAccessEnabled(true);
Mounting the temp. directory: Note that on MacOS you may have to run TMPDIR=/private$TMPDIR docker-compose up if
$TMPDIR contains a symbolic link that cannot be mounted by Docker.
(See details here: https://bitbucket.org/atlassian/localstack/issues/40/getting-mounts-failed-on-docker-compose-up)
If you run into file permission issues on pip install under Mac OS (e.g., Permission denied: '/Library/Python/2.7/site-packages/six.py'), then you may have to re-install pip via Homebrew (see this discussion thread). Alternatively, try installing
with the --user flag: pip install --user localstack
If you are deploying within OpenShift, please be aware: the pod must run as root, and the user must have capabilities added to the running pod, in order to allow Elasticsearch to be run as the non-root localstack user.
The environment variable no_proxy is rewritten by LocalStack.
(Internal requests will go straight via localhost, bypassing any proxy configuration).
If you pull the repo in order to extend/modify LocalStack, run this command to install all the dependencies:
install
This will install the required pip dependencies in a local Python virtualenv directory
.venv (your global python packages will remain untouched), as well as some node modules
in ./localstack/node_modules/. Depending on your system, some pip/npm modules may require
additional native libs installed.
The Makefile contains a target to conveniently run the local infrastructure for development:
infra
Check out the developer guide which contains a few instructions on how to get started with developing (and debugging) features for LocalStack.
The project contains a set of unit and integration tests that can be kicked off via a make target:
test
The projects also comes with a simple Web dashboard that allows to view the deployed AWS components and the relationship between them.
lstack web
pipenv dependency issue; Docker JUnit test runner; POJO type for Java Lambda RequestHandler; Java Lambda DynamoDB event; reuse Docker containers for Lambda invocations; API Gateway wildcard path segments; fix SNS RawMessageDeliverydocker.for.mac.localhost to connect to LocalStack from Docker on Mac; fix b64 encoding for Java Lambdas; fix path of moto_server commandGenericProxyHandler; add $PORT_WEB_UI and $HOSTNAME_EXTERNAL configs; API Gateway path parameters; enable flake8 linting; add config for service backend URLs; use ElasticMQ instead of moto for SQS; expose $LOCALSTACK_HOSTNAME; custom environment variable support for Lambda; improve error logging and installation for Java/JUnit; add support for S3 REST Object POSTUSE_SSL config); create Docker base image; fix issue with DATA_DIRWe welcome feedback, bug reports, and pull requests!
For pull requests, please stick to the following guidelines:
Please note that by contributing any code or documentation to this repository (by raising pull requests, or otherwise) you explicitly agree to the Contributor License Agreement.
This project exists thanks to all the people who contribute.
Thank you to all our backers! ? [Become a backer]
Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [Become a sponsor]
Copyright (c) 2017 LocalStack maintainers and contributors.
Copyright (c) 2016 Atlassian and others.
This version of LocalStack is released under the Apache License, Version 2.0 (see LICENSE.txt). By downloading and using this software you agree to the End-User License Agreement (EULA).
We build on a number of third-party software tools, with the following licenses:
Third-Party software | License —————————-|———————– Python/pip modules: | airspeed | BSD License amazon_kclpy | Amazon Software License boto3 | Apache License 2.0 coverage | Apache License 2.0 docopt | MIT License elasticsearch | Apache License 2.0 flask | BSD License flask_swagger | MIT License jsonpath-rw | Apache License 2.0 moto | Apache License 2.0 nose | GNU LGPL pep8 | Expat license requests | Apache License 2.0 subprocess32 | PSF License Node.js/npm modules: | kinesalite | MIT License Other tools: | Elasticsearch | Apache License 2.0