Name: mkto-pd-sync-app
Owner: Nuxeo
Description: Flask application that synchronizes data between Marketo and Pipedrive.
Created: 2017-02-16 14:52:46.0
Updated: 2018-03-16 14:14:00.0
Pushed: 2018-03-16 14:13:59.0
Size: 964
Language: Python
GitHub Committers
| User | Most Recent Commit | # Commits |
|---|
Other Committers
| User | Most Recent Commit | # Commits |
|---|
MKTO-PD Sync App is a Flask application running on a Google App Engine that synchronizes data between marketing software Marketo and CRM Pipedrive.
The application handles user requests asynchronously using task (push) queues.
app.yaml Application queue.yaml worker.yaml
+ enqueues task + +
| + | |
+----------+----------+ | +---------+----------+ +----------+----------+
| | | | | | |
| Application | | | Task Queue Service | | Worker |
TP POST | | | | | HTTP POST | |
--+----->+ +-----------------+ +-----+---->+ +----------------+ +---------->+ +-----------------+ |
| | |Flask Application| | | | Push Queue | | | |Flask Application| |
| | +-----------------+ +<----+-----+ +----------------+ +<----+-----+ +-----------------+ |
| +---------------------+ | +--------------------+ | +---------------------+
| | | |
+ + + +
calls API Returns task Returns Worker executes task
name and ETA sync
status
Simple Python client for the Marketo REST API.
Lead, opportunities, roles and companies are classes that can be instantiated given a Marketo client instance.
All fields are loaded and available for reading and a defined subset for each entity is available for updating.
An entity also has an “id field” that will be used to match an existing entity.
An existing entity can be loaded given an identifier to the constructor and an eventual id field to look for different from its actual one which can be any exiting field for leads but should be a “searchable field” for opportunities, roles and companies.
An entity can be saved: it is then created or updated depending on if it matches an existing one.
An entity can be deleted.
rt sync.marketo as marketo
= marketo.MarketoClient(IDENTITY_ENDPOINT, CLIENT_ID, CLIENT_SECRET, API_ENDPOINT)
t a lead by id
= marketo.Lead(mkto, "12345")
t a lead by email
= marketo.Lead(mkto, "lead@mail.com", "email")
date lead data
.firstName = "New Name"
.save()
lete lead
.delete()
Simple Python client for the Pipedrive REST API.
People, organizations, deals and activities are classes that can be instantiated given a Pipedrive client instance.
All fields are loaded and available for reading and updating.
An existing entity can be loaded given an identifier or a name if the “id field” parameter is given as “name”. A lead can also be loaded given a custom filter a the “id field”. Currently supported filters are “email domain” and “marketo id”.
Custom fields can be read using the snake case display name rather than their hash key.
Some entities are related to other and can be loaded as well if so.
An entity can be saved: it is then created or updated depending on if it matches an existing one.
For related entities, only the association can be updated (no cascade update).
An entity can be deleted.
rt sync.pipedrive as pipedrive
pipedrive.PipedriveClient(PD_API_TOKEN)
t a person by id
on = pipedrive.Person(pd, "12345")
t an organization by name
nization = pipedrive.Organization(pd, "MyCompany", "name")
t a person custom field with its nicer name
on.lead_score # or getattr(person, {HASH KEY})
t a person related organization
nization = person.organization
date person data
on.name = "New Name"
on.save()
lete person
on.delete()
All requests to the API must be made over SSL (https not http) and validated against the prod key (see configuration).
/marketo/lead/<int:lead_id>: to send lead data from Marketo to PipedriveActually synchronizes data if and only if it is new or it has changed.
/pipedrive/person/<int:person_id> (or /pipedrive/person for Pipedrive notification usage): to send person data from Pipedrive to MarketoActually synchronizes data if and only if it is new or it has changed.
/pipedrive/organization/<int:organization_id> (or /pipedrive/organization for Pipedrive notification usage): to send organization data from Pipedrive to MarketoActually synchronizes data if and only if it is new or it has changed.
/pipedrive/deal/<int:deal_id> (or /pipedrive/deal for Pipedrive notification usage): to send deal data from Pipedrive to MarketoActually synchronizes data if and only if it is new or it has changed.
POST: /marketo/lead/<int:lead_pipedrive_id>/delete: to delete a person in Pipedrive (using lead pipedrive id)
POST: /pipedrive/person/<int:pipedrive_marketo_id>/delete (or /pipedrive/person/delete for Pipedrive notification usage): to delete a lead in Marketo (using person marketo id)
POST: /marketo/lead/<int:lead_id>/activity : to send an activity to Pipedrive from Marketo lead data. It allows a parameter type which if set to email_sent send activities to Pipedrive from the latest Emails sent the same day in Marketo.
POST: /pipedrive/organization/<int:organization_id>/compute (or /pipedrive/organization/compute for Pipedrive notification usage): to compute organization data in Pipedrive
POST: /pipedrive/deal/notify (for Pipedrive notification usage only): to notify about a deal (status change and note added) in Slack
A mapping file describes how Marketo and Pipedrive entity fields should be matched.
ING_NAME = {
to_field_key_1: {
"fields": [from_field_key_1, [from_field_key_2...]],
["mode": "join"/"choose",]
["pre_adapter": function,]
["post_adapter": function]
},
to_field_key_2: {
"transformer": function
}
HOME that targets the folder where you extracted the SDK (see also).Assuming you already have Python 2.7 installed:
lib/ folder:install -r requirements.txt -t lib/
instance folder in the root folder of the project.config.py.sample file to this folder (remove the .sample extension) and set:IDENTITY_ENDPOINT, CLIENT_ID, CLIENT_SECRET, API_ENDPOINT from your Marketo credentials,PD_API_TOKEN from your Pipedrive API token,FLASK_AUTHORIZED_KEYS with any keys (one for unit testing and the other for production).SLACK_WEBHOOK_URL and DATADOG_API_KEYDEBUG and TESTING are logging control variables.
appserver.py -A sync-app app.yaml worker.yaml # Start the local development server
Marketo and Pipedrive clients are tested in the tests module but should not be continuously run as they actually call Marketo and Pipedrive APIs.
The application is also tested in this module where calls to the APIs have been mocked (see files in the resources folder under tests).
Launch the test:
rt GOOGLE_APP_ENGINE={PATH_TO_YOUR_GOOGLE_SDK}/platform/google_appengine
on -m tests.test_sync
To run the test in Eclipse, you must change the Working directory to Default (see this screenshot) and set an environment variable for GOOGLE_APP_ENGINE pointing to the directory where you extracted the SDK (see this other screenshot) in your configuration.
To run a single unit test in Eclipse you can use the keybinding CTRL + F9 while the file open.
You can upload the application running the following command from within the root directory of the project (don't forget the config.py file):
ud app deploy app.yaml worker.yaml queue.yaml [--project [YOUR_PROJECT_ID]]
or
ud config set project marketo-1041
ud components update
ud app deploy app.yaml worker.yaml queue.yaml
Otherwise, this project is continuously delivered (tested and deployed) with Travis and triggered by a push to the master branch of this repository