Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [geomesa-dev] Kafka Replay problems

Hi Scott,

The good news is that the Kafka replay capability is secondary and isn't required to use the Live mode (which is probably more fun).  That said, let's see if we can get things going! 

As a quick guess, I think there is a networking issue to run down between the two VMs.  Since Accumulo and the Live Kafka quick starts are working, we can conclude that Zookeeper, Accumulo, and Kafka are all working at least basically.  The replay capability has to sort through the Kafka log, and I'm guessing there is something in there is using the wrong hostname or something.  

To troubleshoot it, I'd suggest trying to identify the right slf4j jar to drop into WEB-INF/lib to turn on logging.  In my GeoServer lib directory, I'm seeing slf4j-api-1.5.8.jar and slf4j-log4j12-1.4.2.jar.  If either of those are missing, I'd suggest grabbing a copy and seeing if that resolves in the logging issues.  

Assuming that fixes up log4j/slf4j, I'd change the logging for org.locationtech.geomesa.kafka to 'trace' and trying again.  This file (1) is used to find the Kafka offset to start reading from for a replay.  If there are any further log messages, we may be able to learn something.

Also, how long have you let the replay consumer code try to run?  I imagine that it may time out after a minute or 5 and throw an interesting exception.

I hope that helps; I'll chat with some of the folks who worked on this code on Monday to see if we can come up with some more ideas.

Cheers,

Jim  

1.  https://github.com/locationtech/geomesa/blob/master/geomesa-kafka/geomesa-kafka-datastore/src/main/scala/org/locationtech/geomesa/kafka/consumer/offsets/OffsetManager.scala

----- Original Message -----
From:
"Discussions between GeoMesa committers" <geomesa-dev@xxxxxxxxxxxxxxxx>

To:
"geomesa-dev" <geomesa-dev@xxxxxxxxxxxxxxxx>
Cc:

Sent:
Fri, 17 Jul 2015 18:30:12 -0400
Subject:
[geomesa-dev] Kafka Replay problems


Hello,

  I have the GeoMesa Kafka sample working for the live consumer. When the sample code gets to the replay consumer, it hangs at "Consuming with the replay consumer". The feature layer shows up in GeoServer but when I try to publish that layer, GeoServer just sits there with the spinner going. There aren't any errors in the GeoServer log but I have to restart Tomcat to recover GeoServer for any other tasks (going to other pages, etc.).

  I'm using GeoServer 2.5.2 on CentOS 6.5 with GeoMesa 1.1.0 rc3-SNAPSHOT on one VM and I'm using the HortonWorks Sandbox 2.2 with Accumulo 1.5.2 and Kafka 2.10-8.1 and Scala 2.10. The GeoMesa-GeoServer plugin is working well with Accumulo BTW.

  I am seeing the error about log4j and slf4j. I followed the steps in the GeoMesa documentation but that didn't make those WARNs and Errors go away. Could that be a related problem?

  Any ideas on troubleshooting?

Thanks,

 Scott

Back to the top