ConsenSys/IPFS-Store

Name: IPFS-Store

Owner: ConsenSys

Description: IPFS Storage service with search capability

Created: 2018-02-15 17:51:49.0

Updated: 2018-02-18 01:42:29.0

Pushed: 2018-02-16 20:37:09.0

Homepage: null

Size: 43

Language: Java

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

IPFS-Store (service)

IPFS-Store aims is to provide an easy to use IPFS storage service with search capability for your project.

The service is composed of three main components:

• IPFS-Store which provides an API to store/index/search/fetch
• IPFS Node IPFS client such as js-ipfs or go-ipfs
• ElasticSearch engine (other implementations coming soon)

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

• Java 8
• Maven
• Docker (optional)

Build

1. After checking out the code, navigate to the root directory

    /path/to/ipfs-store/ipfs-store-service/
   

2. Compile, test and package the project

   n clean package
   

3. Run the project

a. If you have a running instance of IPFS and ElasticSearch

Executable JAR:

port IPFS_HOST=localhost
port IPFS_PORT=5001
port ELASTIC_CLUSTERNODES=localhost:9300
port ELASTIC_CLUSTERNAME=elasticsearch

va -jar target/ipfs-store.jar

Docker:

cker build  . -t kauri/ipfs-store:latest

port IPFS_HOST=localhost
port IPFS_PORT=5001
port ELASTIC_CLUSTERNODES=localhost:9300
port ELASTIC_CLUSTERNAME=elasticsearch

cker run -p 8040:8040 kauri/ipfs-store

b. If you prefer build all-in-one with docker-compose

cker-compose -f docker-compose.yml build
cker-compose -f docker-compose.yml up

API Documentation

Overview

| Operation | Description | Method | URI | | ——– | ——– | ——– | ——– | | store | Store content into IPFS |POST | /ipfs-store/store | | index | Index content |POST | /ipfs-store/index | | store_index | Store & Index content | POST | /ipfs-store/store_index | | fetch | Get content | GET | /ipfs-store/fetch/{index}/{hash} | | search | Search content | POST | /ipfs-store/search/{index} | | search | Search content | GET | /ipfs-store/search/{index} |

Details

Store content

Store a content (any type) in IPFS

• URL: /ipfs-store/store
  

• Method: POST

• Header: N/A

• URL Params: N/A

• Data Params:

  • file: [content]

• Sample Request:

  rl -X POST \
  p://localhost:8040/ipfs-store/store' \
  file=@/home/gjeanmart/hello.pdf'
  

• Success Response:

  • Code: 200
    Content:

    
    h": "QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o"
    
    

---

Index content

Index IPFS content into the search engine

• URL /ipfs-store/index
• Method: POST
• Header:
  

| Key | Value | | ——– | ——– | | content-type | application/json |

• URL Params N/A

• Data Params

  • request:

| Name | Type | Mandatory | Default | Description | | ——– | ——– | ——– | ——– | ——– | | index | String | yes | | Index name | | id | String | no | | Identifier of the document in the index. id null, autogenerated | | content_type | String | no | | Content type (MIMETYPE) | | hash | String | yes | | IPFS Hash of the content | | index_fields | Key/Value[] | no | | Key/value map presenting IPFS content metadata|

ndex": "documents", 
d": "hello_doc",
ontent_type": "application/pdf",
ash": "QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o",
ndex_fields": [
{
  "name": "title",
  "value": "Hello Doc"
}, 
{
  "name": "author",
  "value": "Gregoire Jeanmart"
}, 
{
  "name": "votes",
  "value": 10
}, 
{
  "name": "date_created",
  "value": 1518700549
}

• Sample Request:

 -X POST \
'http://localhost:8040/ipfs-store/index' \
-H 'content-type: application/json' \  
-d '{"index":"documents","id":"hello_doc","content_type":"application/pdf","hash":"QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o","index_fields":[{"name":"title","value":"Hello Doc"},{"name":"author","value":"Gregoire Jeanmart"},{"name":"votes","value":10},{"name":"date_created","value":1518700549}]}'

• Success Response:

  • Code: 200
    Content:

    
    ex": "documents",
    : "hello_doc",
    h": "QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o"
    
    

---

Store & Index content

Store content in IPFS and index it into the search engine

• URL /ipfs-store/store_index

• Method: POST

• Header: N/A

• URL Params N/A

• Data Params

  • file: [content]
  • request:

| Name | Type | Mandatory | Default | Description | | ——– | ——– | ——– | ——– | ——– | | index | String | yes | | Index name | | id | String | no | | Identifier of the document in the index. id null, autogenerated | | content_type | String | no | | Content type (MIMETYPE) | | hash | String | yes | | IPFS Hash of the content | | index_fields | Key/Value[] | no | | Key/value map presenting IPFS content metadata|

ndex": "documents", 
d": "hello_doc",
ontent_type": "application/pdf",
ash": "QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o",
ndex_fields": [
{
  "name": "title",
  "value": "Hello Doc"
}, 
{
  "name": "author",
  "value": "Gregoire Jeanmart"
}, 
{
  "name": "votes",
  "value": 10
}, 
{
  "name": "date_created",
  "value": 1518700549
}

• Sample Request:

 -X POST \
tp://localhost:8040/ipfs-store/store_index \
 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
 file=@/home/gjeanmart/hello.pdf \
 'request={"index":"documents","id":"hello_doc","content_type":"application/pdf","hash":"QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o","index_fields":[{"name":"title","value":"Hello Doc"},{"name":"author","value":"Gregoire Jeanmart"},{"name":"votes","value":10},{"name":"date_created","value":1518700549}]}'

• Success Response:

  • Code: 200
    Content:

    
    ex": "documents",
    : "hello_doc",
    h": "QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o"
    
    

---

Get content

Get content on IPFS by hash

• URL http://localhost:8040/ipfs-store/fetch/{index}/{hash}

• Method: GET

• Header: N/A

• URL Params N/A

• Sample Request:

rl \
'http://localhost:8040/ipfs-store/fetch/documents/QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o' \
-o hello_doc.pdf 

• Success Response:

  • Code: 200
    Content: (file)

---

Search contents

Search content accross an index using a dedicated query language

• URL http://localhost:8040/ipfs-store/search/{index}
• Method: GET or POST
• Header:
  

| Key | Value | | ——– | ——– | | content-type | application/json |

• URL Params

| Name | Type | Mandatory | Default | Description | | ——– | ——– | ——– | ——– | ——– | | pageNo | Int | no | 1 | Page Number | | pageSize | Int | no | 20 | Page Size / Limit | | sort | String | no | | Sorting attribute | | dir | ASC/DESC | no | ASC | Sorting direction | | query | String | no | | Query URL encoded (for GET call) |

• Data Params

The search operation allows to run a multi-criteria search against an index. The body combines a list of filters :

| Name | Type | Description | | ——– | ——– | ——– | | name | String | Index field to perform the search | | names | String[] | Index fields to perform the search | | operation | See below | Operation to run against the index field | | value | Any | Value to compare with |

| Operation | Description | | ——– | ——– | | full_text | Full text search | | equals | Equals | | not_equals | Not equals | | contains | Contains the word/phrase | | in | in the following list | | gt | Greater than | | gte | Greater than or Equals | | lt | Less than | | lte | Less than or Equals |

uery": [
{
  "name": "title",
  "operation": "contains",
  "value": "Hello"
},
{
  "names": ["author", "title"],
  "operation": "full_text",
  "value": "Gregoire"
},
{
  "name": "votes",
  "operation": "lt",
  "value": "5"
}

• Sample Request:

 -X POST \
'http://localhost:8040/ipfs-store/search/documents' \
-H 'content-type: application/json' \  
-d '{"query":[{"name":"title","operation":"contains","value":"Hello"},{"name":"author","operation":"equals","value":"Gregoire Jeanmart"},{"name":"votes","operation":"lt","value":"5"}]}'

 -X GET \
ttp://localhost:8040/ipfs-store/search/documents?page=1&size=2&query=%7B%22query%22%3A%5B%7B%22name%22%3A%22votes%22%2C%22operation%22%3A%22lt%22%2C%22value%22%3A%225%22%7D%5D%7D' \
 'Accept: application/json' \
 'Content-Type: application/json' \

• Success Response:

  • Code: 200
    Content:

ontent": [
{
  "index": "documents",
  "id": "hello_doc",
  "hash": "QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o",
  "content_type": "application/pdf",
  "index_fields": [
    {
      "name": "__content_type",
      "value": "application/pdf"
    },
    {
      "name": "__hash",
      "value": "QmWPCRv8jBfr9sDjKuB5sxpVzXhMycZzwqxifrZZdQ6K9o"
    },
    {
      "name": "title",
      "value": "Hello Doc"
    },
    {
      "name": "author",
      "value": "Gregoire Jeanmart"
    },
    {
      "name": "votes",
      "value": 10
    },
    {
      "name": "date_created",
      "value": 1518700549
    }
  ]
}



t": null,
stPage": false,
alElements": 4,
tPage": true,
alPages": 1,
berOfElements": 4,
e": 20,
ber": 1

Clients

Java

Spring-Data

CLI

Javascript

TODO

• Implement clients: Java, Spring-Data, CLI, Javascript, Python
• Authentication (by signature?)
• Integration tests