Re: [geomesa-users] Tips on ingesting a lot of data into GeoMesa

[Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx>]

That converter needs some more work... it generally works fine for smaller data sets, but for larger files (e.g. planet.gpx) it bogs down. If you're interested, it needs to be changed to use an arbitrary JDBC connection for storing the nodes - the in memory/on disk implementations don't hold up. I've actually been holding off because I'm not sure the underlying OSM converter libraries will be approved for use by Eclipse.

Thanks,

Emilio

On 01/27/2017 10:31 AM, Damiano Albani wrote:

Hi Emilio,

On Fri, Jan 27, 2017 at 4:16 PM, Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> wrote:

For large data sets, you might try the NYC Taxi data - they have 1.2B records available for download. I'm not sure how much that would translate to in size on disk though.

Nice tip, I'll have a look at this dataset!

I've also seen your great work to support OpenStreetMap data conversion.

It compiled just fine, now I need to have a go at running it.

Or do I need to know something about the status of this converter?

Best regards,

 

--

Damiano Albani
Geodan

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users