Transparency Toolkit

Login: TransparencyToolkit

Company: null

Location: null

email:

Blog: https://transparencytoolkit.org

Members

  1. Brennan Novak
  2. M. C. McGrath

Repositories

ACLUScraper
Scraper for ACLU court documents.
ansible-role-lookingglass
Automates deployment of LookingGlass instances
ArchivePile
A read-only theme for publishing email archives using Mailpile
Archiver
Archives URLs
Catalyst
null
classification-sensation
Parse classification-related information
congressionalrecord
A scraper for the congressional record.
CountryConvert
Converts 2-char ISO country codes to 3-char codes.
CrawlerManager
API for calling crawlers
DataPolitics
Data on the political marketing industry
dataspec-EmailCrawl
Dataspec for emails
dataspec-fbidhs
null
dataspec-GoogleCrawl
A dataspec for the Google crawler
dataspec-IndeedCrawl
A LookingGlass dataspec file for data scraped from Indeed.com
dataspec-LinkedinCrawl
A LookingGlass dataspec file for data scraped form LinkedIn.com
dataspec-LoadFiles
Dataspec for plain files loaded in via Harvester/DirCrawl.
dataspec-sii
Dataspec for SII
dataspec-snowden
A dataspec for Snowden documents
dataspec-template
A starter template for LookingGlass json files
dataspec-TsjobCrawl
Dataspec for cleared job listings
dataspec-TwitterCrawl
LookingGlass dataspec for tweets
DesignAssets
A collection of branding, interfaces, and other visual resources!
DirCrawl
Runs block of code on every file in directory
DocIntegrityCheck
Methods for encrypting and verifying documents. Utility gem for document processing pipeline.
DocManager
Universal backend for indexing, storing, and querying documents.
DocSuggestions
Backend for processing document suggestions from LookingGlass
DocUpload
Upload application for documents in archiving service.
EFFScraper
A scraper for EFF court case documents.
EmailParser
A crawler for converting email files on disk to JSON
EncryptionTerrorism-Data
A collection of public data about the debate of encryption helping terrorists
EntityExtractor
Extracts entities and terms matching certain patterns.
ExtractPatterns
Extracts terms matching certain patterns. For finding new codewords and tracking mentions of known ones.
FacebookCrawler
A crawler for Facebook data from public web and Graph API
federalregisterscraper
Scraper for the Federal Register
generalscraper
Scrapes all pages on any site you specify for keywords.
gj-docs
null
Harvester
Web crawling and document processing through a usable interface.
HarvesterReporter
Incremental crawler result reporting for Transparency Toolkit
IC-Company-Data
Intelligence contractors
ICWATCH-Data
Resume data and scripts for managing it
IndeedCrawler
Crawler for the resume website Indeed
IndeedParser
Parser for Indeed resumes
IndeedScraper
Scraper for Indeed
JSONCombiner
Combines JSONs.
JSONCrossreference
Crossreferences JSONs and returns the matching data.
JSONToChart
Converts JSONs to pretty charts
JSONToChoropleth
Generates choropleth maps from JSONs.
JSONToMap
Converts a JSON with locations into a map with points.
JSONToNetworkGraph
Generates network graphs from a JSON.
LinkedinCrawler
Crawls public LinkedIn profiles
LinkedInData
Scrapes all LinkedIn profiles including search terms.
LinkedinParser
A parser for LinkedIn profiles
LookingGlass
Intuitive and configurable search interface for document archives.
month-names
Names of months in multiple languages
NameToEmail
Gets a list of potential emails from a JSON with names.
NetworkGraph
Neo4j network graph generator prototype
NSA-Data
NSA documents in machine readable form
OCRServer
OCR server for hosted archiving service
ParseFile
OCRs document and extracts metadata
PiplCollector
Request info from Pipl for all items in dataset
PiplRequest
Request profiles from Pipl
RandomCode
Random, non-generalized bits of code
RequestManager
Manages scraper HTTP requests
SIGADParse
All your SIGAD are belong to us
sunlightcongress
Ruby API wrapper for Sunlight Foundation's congress data.
sunlightpartytime
Ruby API wrapper for Sunlight Foundation's Party Time data.
Surveillance-Research-Data
Raw data and scripts for Surveillance Research Archive
Test-Data
Test data for Transparency Toolkit development
theme-pi
A theme for Privacy International collaborations
theme-snowden
A theme for LookingGlass for Snowden doc search
Thumbtack
An open narrative mapping tool to corroborate narratives across multiple sources and formats
TransparencyToolkit
Main repository for Transparency Toolkit
transparencytoolkit.github.io
A styleguide site for Transparency Toolkit
Transparency-Toolkit-Prototype
Analysis system for Transparency Toolkit.
TSJobCrawler
Collects listings for jobs that require security clearance.
tt-ansible
Ansible roles for deployment. In development, expect problems.
ttcalc
Calculation functions for Transparency Toolkit.
Twiddler
A user friendly tool for text processing, light NLP, and keyword extraction
TwitterCrawler
A crawler for Twitter
UDPServer
Manages communications over UDP between different parts of the pipeline
UNVoteScrape
Scraper for UN GA Vote Records
UploadConvert
Tools for converting documents uploaded to Transparency Toolkit to properly formatted JSONs.
UtilityScripts
Scripts for managing scrapers
wlsearchscraper
Gets a list of results from the WikiLeaks search.
wordcloud
Changes word sizes in a document based on the number of times they occur.
This work is supported by the National Institutes of Health's National Center for Advancing Translational Sciences, Grant Number U24TR002306. This work is solely the responsibility of the creators and does not necessarily represent the official views of the National Institutes of Health.