Name: Social-Media-Combine
Owner: NCSU Libraries
Description: This project is no longer supported. A pre-configured collection of tools including Social Feed Manager and Lentil for easily building Twitter and Instagram social media archives on your own computer.
Created: 2015-06-10 18:19:17.0
Updated: 2018-02-09 15:35:49.0
Pushed: 2018-02-09 12:53:08.0
Size: 48
Language: JavaScript
GitHub Committers
User | Most Recent Commit | # Commits |
---|
Other Committers
User | Most Recent Commit | # Commits |
---|
The Social Media Combine has been deprecated and is no longer supported.
On January 30th, 2018, Instagram announced the release of a new Instagram Graph API in order to focus on supporting business and advertising use of Instgram content. The Instagram Graph API does not support the use cases required by Lentil. Instagram immediately removed their mechanism for requesting developer credentials for the older Instagram API and it is no longer possible to create new instances of Lentil. The older Instagram API will be deprecated over the next two years. Lentil will continue to work with pre-existing developer credentials until December 11, 2018. Due to the deprecation of Lentil, the Social Media Combine project has also been deprecated and is no longer supported.
The Social Media Combine makes developing collections of social media much easier. It pre-assembles Lentil (for Instagram data harvesting) and Social Feed Manager (for Twitter data harvesting), along with the web servers and databases needed for their use, into a single package that can be deployed to desktop and laptop computers and used in Windows, OSX, or Linux. We believe in the documentary value of social media archives and hope that the Social Media Combine will make it possible to quickly collect social media data on everyday hardware.
Install Git
Install Vagrant version 1.7.4 or greater on your host machine.
Open Terminal
on a Mac, Git Shell
on Windows, or your preferred shell on Linux and clone this repository using git.
clone https://github.com/NCSU-Libraries/Social-Media-Combine.git
ocial-Media-Combine
You can run Combine on VM (step 4.1) OR Combine without VM (directly on Docker)(step 4.2).
4.1. Combine on VM
Install Virtualbox
Run vagrant up
from within the Social-Media-Combine directory.
4.2. Combine without VM [tested on Ubuntu 14.04]
Install Docker version 1.8.1 or greater
Run Docker Daemon on public interface
Append following to the /etc/default/docker file
```
DOCKER_OPTS="-H <ip>:<port>"
```
Restart docker service
```
service docker restart
```
Define DOCKER_HOST and VAGRANT_NO_PARALLEL environment variables
```
export DOCKER_HOST=<ip>:<port>
export VAGRANT_NO_PARALLEL=true
```
Run
agrant up --docker=true
If working on Windows and
vagrant up
immediately returns errors, you may need to runsh eol_to_unix.sh
This opens a web browser with configuration form. If it doesn't, visit http://localhost:8081 in your browser. Make changes and click OK.
Enter configuration parameters in the web configuration form and click OK. Your applications should be updated within a minute or so.
Visit:
vagrant ssh
on the command line and then the twarc.py
commands described in the official twarc documentation. Store output files in the
/vagrant/archive/twarc-store
directory.To stop the Social Media Combine virtual machine, run
vagrant halt
in the project directory.
vagrant provision
.After you have been harvesting content for a day or so, take a look in the archive
directory. Some Social Feed Manager content may not yet appear in this directory, but will still be present in the database.
To generate a CSV export of data collected for a Twitter user (e.g. @cazzerson) in Social Feed Manager, you can run vagrant exec 'docker exec vagrant_sfmapp_1 ./sfm/manage.py export_csv --twitter-user cazzerson'
. You can also add users to a set (e.g. myset) in the Social Feed Manager administrative tools and then export this set of users with vagrant exec 'docker exec vagrant_sfmapp_1 ./sfm/manage.py export_csv --set-name myset'
.
Lentil:
Social Feed Manager:
Application reconfiguration can be performed after deployment.
Open web configuration tool at http://localhost:8081. Make changes and click OK.
Your applications should be updated within a minute or so.
If your applications are not updated or you are changing configuration files on the filesystem, you may need to manually launch the configuration task by running
vagrant exec ./reconfig.sh
.
ant provision
Please email Bret Davidson at bddavids@ncsu.edu with any questions or suggestions.
The development of the Social Media Combine environment was partially supported by federal Library Services and Technology Act (LSTA) funds made possible through a grant from the Institute of Museum and Library Services, and administered by the State Library of North Carolina, a division of the Department of Cultural Resources. This grant-funded project, “New Voices and Fresh Perspectives,” is hosted by NCSU Libraries and led by Brian Dietz and Jason Casden.
Social Feed Manager is developed at George Washington University Libraries.
Lentil is developed at NCSU Libraries.
twarc is developed by Ed Summers.