Re: [geomesa-users] Tips on ingesting a lot of data into GeoMesa

[Jim Hughes <jnh5y@xxxxxxxx>]

Hi all,

Good news on this front:  the Eclipse board has approved OSM4J and its dependencies as ok for use in LocationTech projects!  Emilio, sorry for the long fuse on getting that sorted out.  When you have time, feel free to update the PR and improve it!

Cheers,

Jim

On 01/27/2017 10:46 AM, Emilio Lahr-Vivaz wrote:

That converter needs some more work... it generally works fine for smaller data sets, but for larger files (e.g. planet.gpx) it bogs down. If you're interested, it needs to be changed to use an arbitrary JDBC connection for storing the nodes - the in memory/on disk implementations don't hold up. I've actually been holding off because I'm not sure the underlying OSM converter libraries will be approved for use by Eclipse.

Thanks,

Emilio

On 01/27/2017 10:31 AM, Damiano Albani wrote:

Hi Emilio,

On Fri, Jan 27, 2017 at 4:16 PM, Emilio Lahr-Vivaz <elahrvivaz@xxxxxxxx> wrote:

For large data sets, you might try the NYC Taxi data - they have 1.2B records available for download. I'm not sure how much that would translate to in size on disk though.

Nice tip, I'll have a look at this dataset!

I've also seen your great work to support OpenStreetMap data conversion.

It compiled just fine, now I need to have a go at running it.

Or do I need to know something about the status of this converter?

Best regards,

 

--

Damiano Albani
Geodan

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users

_______________________________________________
geomesa-users mailing list
geomesa-users@xxxxxxxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.locationtech.org/mailman/listinfo/geomesa-users