MarkESmith
(Mark E Smith)
November 25, 2019, 7:39pm
1
Hey all,
I've had no luck using either the command line or GUI ETL tool for pulling and mapping data directly from a Db2 database on an IBM i server.
Despite this, the possibility occurred to me of using the Neo4j ETL tool to query against a Microsoft SQL
linked server. Essentially, setting up a 'linked server' in Microsoft SQL allows you to query a different database (other than SQL Server) and the SQL Server is essentially acting as a pass-through mechanism.
Has anyone tried this before? Would it be possible? I'm wondering if perhaps my luck with the ETL tool will increase if I'm querying the data through a Microsoft SQL server.
fred
(Fred)
November 29, 2019, 10:07am
2
Hey Mark, I have done this agains SQL in the ETL tool and APOC procedures. You have a bit more control using the procedures. I have never used it against db2, have you tried the procedures on the DB2 connection?
procedure
apoc.load.driver
apoc.load.driver(driverClass :: STRING?) :: VOID
apoc.load.driver('org.apache.derby.jdbc.EmbeddedDriver') register JDBC driver of source database
procedure
apoc.load.jdbc
apoc.load.jdbc(jdbc :: STRING?, tableOrSql :: STRING?, params = :: LIST? OF ANY?, config = {} :: MAP?) :: (row :: MAP?)
apoc.load.jdbc('key or url','table or statement', params, config) YIELD row - load from relational database, from a full table or a sql statement
procedure
apoc.load.jdbcParams
apoc.load.jdbcParams(jdbc :: STRING?, sql :: STRING?, params :: LIST? OF ANY?, config = {} :: MAP?) :: (row :: MAP?)
deprecated - please use: apoc.load.jdbc('key or url','',[params]) YIELD row - load from relational database, from a sql statement with parameters
procedure
apoc.load.jdbcUpdate
apoc.load.jdbcUpdate(jdbc :: STRING?, query :: STRING?, params = :: LIST? OF ANY?, config = {} :: MAP?) :: (row :: MAP?)
apoc.load.jdbcUpdate('key or url','statement',[params],config) YIELD row - update relational database, from a SQL statement with optional parameters
procedure
apoc.model.jdbc
apoc.model.jdbc(jdbc :: STRING?, config = {} :: MAP?) :: (nodes :: LIST? OF NODE?, relationships :: LIST? OF RELATIONSHIP?)
apoc.model.jdbc('key or url', {schema:'', write: <true/false>, filters: { tables: , views: , columns: }) YIELD nodes, relationships - load schema from relational database
Did you raise an issue about your DB2 server problem?
MarkESmith
(Mark E Smith)
December 2, 2019, 3:44pm
4
@michael.hunger Yes, the GH issue can be found at the following link:
opened 08:18PM - 07 Nov 19 UTC
I am running the Neo4j Desktop v1.2.1 and Neo4j ETL Tool v1.4.2 on a Windows 10 … machine. I have a remote instance of Neo4j running on a Linux machine (Neo4j v3.5.6) and I have a dB2 database running on an IBM i machine.
I am using a JDBC to connect to the dB2 database and 'Test Connection' results in success (though it does appear to register my username as the schema, even though I am specifying a default schema that is not my username) I am trying to use the ETL tool to map metadata and though the connection appears to be successful, the output (in the logs) is empty. I have attached a picture of my JDBC setup and the resulting logs (when I hit 'Start Mapping'). Any help or tips or questions for more information would be appreciated!
![neo4j_github_dsn_example](https://user-images.githubusercontent.com/9221355/68424054-35934e80-0171-11ea-99ee-85e8c7b204b3.png)
```
COMMAND: java -cp "C:\Users\User\.Neo4jDesktop\graphApps\_global\neo4j-etl-ui/dist/neo4j-etl.jar;C:\Program Files\IBM\SDPShared\plugins\com.ibm.datatools.db2_2.2.201.v20151119_1817\driver\db2jcc_license_cisuz.jar;C:\Program Files\IBM\SDPShared\plugins\com.ibm.datatools.db2_2.2.201.v20151119_1817\driver\db2jcc4.jar" org.neo4j.etl.NeoIntegrationCli generate-metadata-mapping --rdbms:url "jdbc:db2://192.168.1.3:446/S1031056" --rdbms:password "password" --rdbms:user "user" --schema "LSCPRDLIB" --output-mapping-file "C:\Users\USER\AppData\Local\Temp/db2_S1031056_LSCPRDLIB_mapping.json"
- Skipping reading import options from file because file [] doesn't exist.
- Creating RDBMS to CSV mappings...
Available schema: #CGULIB
.....(*ABRIDGED FOR BREVITY)
Available schema: ZMARIADB
- Using database plugin for
- Crawling schemas
- Retrieving all schemas
- Retrieving all catalogs
- Processed 597 rows for <retrieveAllSchemas>
- Including schema LSCPRDLIB
- Retrieved 1 schemas
- Including schema LSCPRDLIB
- Total time taken for <crawlSchemas> - 00:00:00.063 hours
- 98.4% - 00:00:00.062 - <retrieveSchemas>
- 1.6% - 00:00:00.001 - <sortAndFilterSchemas>
- Retrieving database information
- Not retrieving additional database information, since this was not requested
- Retrieving JDBC driver information
- Not retrieving additional JDBC driver information, since this was not requested
- Retrieving SchemaCrawler crawl information
- Total time taken for <crawlDatabaseInfo> - 00:00:00.011 hours
- 0.0% - 00:00:00.000 - <retrieveDatabaseInfo>
- 0.0% - 00:00:00.000 - <retrieveAdditionalDatabaseInfo>
- 0.0% - 00:00:00.000 - <retrieveJdbcDriverInfo>
- 0.0% - 00:00:00.000 - <retrieveAdditionalJdbcDriverInfo>
-100.0% - 00:00:00.011 - <retrieveCrawlHeaderInfo>
- Crawling column data types
- Retrieving system column data types
- Not retrieving user column data types, since this was not requested
- Total time taken for <crawlColumnDataTypes> - 00:00:00.024 hours
-100.0% - 00:00:00.024 - <retrieveSystemColumnDataTypes>
- 0.0% - 00:00:00.000 - <retrieveUserDefinedColumnDataTypes>
- Crawling tables
- Retrieving tables
- Retrieving tables for schema <LSCPRDLIB>
- Processed 3704 rows for <retrieveTables>
- Retrieved 0 tables
- Crawling routines
- Retrieved 0 routines
- Not retrieving synonyms, since this was not requested
- Not retrieving sequences, since this was not requested
OUTPUT:
[ ]
```
MarkESmith
(Mark E Smith)
December 2, 2019, 3:47pm
5
@fred where exactly would you use these procedures? I'm using the Neo4j ETL GUI tool - is there a way to use them within that or are you using it within the ETL CLI tool?