Load data from impala to neo4j

apoc
cypher

(12kunal34) #1

Hey Everyone,

i need some help in importing the data from impala to neo4j .
i have credentials of impala and i want my data in neo4j . could you all please help me and tell me the steps how i can do it.
i have millions of records in my tables and i know there is a apoc procedure apoc.load.jdbc
but i dont know how to use it. i had a look into APOC document as well but no clear understanding for me
please help

Thanks in advance.


(Benoit Simard) #2

Hi,

Firstly you need to put the JDBC driver of impala into the plugin directory of Neo4j, and restart it.

Then, register the driver in neo4j with this query :
CALL apoc.load.driver("com.cloudera.impala.jdbc41.Driver");

And now you can use the apoc JDNC procedure like that :
CALL apoc.load.jdbc("jdbc:impala://MY_SERVER_IP:PORT/MY_DB","SELECT * FROM table" YIELD row ...


(12kunal34) #3

Hi Benoit,

thanks for your reply.
may i know please from where i can download this driver ??
is it opensource ??


(Benoit Simard) #4

Here : https://www.cloudera.com/downloads/connectors/impala/jdbc/2-6-4.html


(12kunal34) #5

Hi Benoit,

i tried as you suggested
but getting below error.

Neo.ClientError.Procedure.ProcedureCallFailed: Failed to invoke procedure `apoc.load.jdbc`: Caused by: java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Client

i am using below query

CALL apoc.load.jdbc("jdbc:impala://internal-edl-dev-ifgfgh-2.elb.amazbgff.com:21050/publish_test","SELECT * FROM student") YIELD row
RETURN row

please help me in this


(Benoit Simard) #6

Can you download this version of the driver instead : https://www.cloudera.com/downloads/connectors/impala/jdbc/2-5-45.html


(12kunal34) #7

Is this because of impala driver ??
Do we need more jars for this like thrift,metastore, hive service ,http client etc ??


(12kunal34) #8

Hey Benoit,

Getting below error now :frowning:

Neo.ClientError.Procedure.ProcedureCallFailed: Failed to invoke procedure `apoc.load.jdbc`: Caused by: org.apache.thrift.transport.TTransportException

one more question
where i need to set my username and password for impala ??

please help me


(Benoit Simard) #9

Yes it seems that the JDBC jar requires some additionals libraries ... byt I have found any trace of that in the cloudera documentation ....

But the version 2.5.45 includes a lot of jars ... that's why I have asked you to try it.


(12kunal34) #10

@Benoit ,

i ahve added all these jars from this version
after adding these jar my error messages is as in above comment.

could you please let me know what am i missing here .


(12kunal34) #11

Hi @stefan.armbruster @andrew.bowman @michael.hunger

Couuld you please help me in above issue
i need this for mu POC.


Create dynamic Json with neo4j path nodes