Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-users] Tips on ingesting a lot of data into GeoMesa
Damn it, I was sure to have paid attention to this... but apparently not: indeed, you're right, hbase-site.xml was missing in the conf/ directory. [facepalm]
Silly me, really...
At least someone in the future might learn from it while reading the archives of the mailing-list.
Huge thanks for your help, Emilio!
Now I can finally play with the ingester and see how it performs / fits into my needs.
I will probably have further questions about it after some use in practice.

On Tue, Jan 24, 2017 at 11:31 PM, Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> wrote:
Did you verify that your hbase-site.xml is on the classpath?


On 01/24/2017 05:09 PM, Damiano Albani wrote:
Running 'geomesa-bigtable classpath' confirms that it's only picking JARs from the lib/ directory.
And, as I wrote previously, I did compare your list of JARs with my own lib/ directory: the only difference is, apart from 1.3.0-m3 vs. 1.3.0, the absence of hbase-client and hbase-common in my situation.
So, I don't understand why you don't run into the same Zookeeper issue as I do.

On Tue, Jan 24, 2017 at 10:59 PM, Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> wrote:
You should be able to run 'geomesa-bigtable classpath' to examine exactly what is being picked up.


On 01/24/2017 04:56 PM, Emilio Lahr-Vivaz wrote:
Did you compare the jars in your lib folder with the list I sent? I know that list works... you might want to clear your lib folder and re-extract it, in case you still have some shaded jars from the GeoMesa HBase distro.

Do you possibly have an HBASE_HOME or HADOOP_HOME defined? It might be picking up configurations from there if so.

On 01/24/2017 04:43 PM, Damiano Albani wrote:
On Tue, Jan 24, 2017 at 10:02 PM, Damiano Albani <damiano.albani@xxxxxxxxx> wrote:

Yes, I do have the Cloud SDK installed and configured.
And I have a Maven-based project of mine which can connect just fine to my BigTable instance.
So I don't understand why it doesn't via geomesa-bigtable...
I'll continue investigating.

Further test: calling the BigTableRunner class does also work fine from IntelliJ.
Strange, maybe some kind of wrong JARs in the classpath?!
I've add a look at the content of geomesa.log which is generated by geomesa-bigtable.
I saw things which were not shown on the console, e.g.:
2017-01-24 21:08:27,134 WARN  [org.apache.hadoop.hbase.shaded.org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.hadoop.hbase.shaded.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2017-01-24 21:08:27,243 WARN  [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.hadoop.hbase.shaded.org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2017-01-24 21:08:28,241 WARN  [org.apache.hadoop.hbase.shaded.org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

I don't get why it's trying to connect to localhost?!

By the way, I've solved my Guava related problem by using the shaded version of the HBase client, as proposed in HBASE-14963. This way, I only had to drop a single JAR in lib/.

--
Damiano Albani
Geodan


_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users
_______________________________________________ geomesa-users mailing list geomesa-users@xxxxxxxxxxxxxxxx To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.locationtech.org/mailman/listinfo/geomesa-users
--
Damiano Albani Geodan 
_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users



--
Damiano Albani
Geodan

Back to the top