Geo-Assist – An in-memory spatial engine to store and query spatial data

https://github.com/thegeekyasian/geo-assist

by thegeekyasian • 3 years ago

50 20 3 years ago

I've created a project that provides in-memory Geo-spatial Indexing, with 2-dimensional K-D Tree. I've worked on multiple projects where I used K-D Trees to find the nearest neighbors for provided geo coordinates with efficient results. I used this implementation for applications with ~35m requests per day, with custom modifications and it worked like a charm. Hence I have created a generic implementation, ready to use. I have already shared great feedback from many programmers and working on improvements already to add new features. The project is open to suggestions and contributions as well. Feel free to create any issues on the GitHub repository.

nonethewiser2• 3 years ago

This is a total aside from someone who doesn't know Java...

But why design Point so that you have to do:

`Point point = new Point.Builder() .latitude(25.2012544) .longitude(55.2569389) .build();`

Instead of:

`Point point = new Point(25.2012544,55.2569389)`

Cool project btw... working with geospatial data can be fun.

iampims• 3 years ago

Less options for mixing latitude and longitude?

I personally like static methods.

Point point = Point.fromLonLat(55.25…, 25.20…);

The builder pattern is useful to set defaults and add features without breaking backward compatibility.

bfeynman• 3 years ago

builder patterns are very common now because they make maintainability a lot easier, (i.e. if god forbid someone accidentally swapped vars in list of inputsof the same type), and secondly because you can use an annotation library to generate the boilerplate for you, you don't have to write any of it.

Although agreed it's used when you start to get to more parameters. I didn't check but it's possible there are a lot of null/default values like projection or something.

just_mc• 3 years ago

Builder patterns are most useful in statically typed languages that don't have strong support for optional parameters.

In creating a Point, neither value is optional, so I would question the value versus the additional complexity in this case.

asadawadia2• 3 years ago

thanks - I really wanted a library like this on the JVM

`kdTree.findNearestNeighbor(point, 2);` does this return

1. A list of ALL points that match the distance predicate?

2. Are they sorted by distance to the target?

3. Is it possible to cap to only the closest N points?

thegeekyasian2• 3 years ago

I am working on the similar feature Asad You can connect with me on the discord server for geo-assist https://discord.gg/8Xe2Ds4BWj

I would like to have your input and have the better understanding of what you are looking for as well.

rolph• 3 years ago

now do this in 3d, and forget about the geolocation, move toward establishing location and orientation of limb segments, and establish a set of vectors that will satisfy a required velocity/distance intercept, to coordinate limb function and interaction, then worry about where on the globe, the local instance is with respect to other nearby instances; that would impress me, and would be much more usefull than nearest nieghbours geolocation.

jillesvangurp• 3 years ago

There are a lot of database/search products that allow you to do geospatial queries. Stuff like this is nice if you are implementing your own version of that but using it as part of a regular web server or batch toolset is probably a bad idea unless your data set is small.

Some examples:

- redis has support for geospatial. Good for radius searches.

- elasticsearch/opensearch have pretty elaborate options for disance search, ranking, aggregating, etc. It can also deal with geojson geometry and do polygon intersection/containment queries, etc.

- mongodb has similar features for geojson but is probably a bit more limited for querying/ranking/aggregating.

- postgis is a bit of a beast but a goto solution for doing complex geospatial work.

There are many more options of course. Just making the point that this isn't a wheel that is in a lot of need of reinventing.

zX41ZdbW• 3 years ago

ClickHouse has Geo data types and functions (such as import and export from WKT, GeoJSON) and special features to accelerate geospatial queries.

For example, polygonal dictionaries allow doing coordinate-to-region lookups using the indexed data structure. It is used for mobile app analytics and ride-sharing apps.

https://clickhouse.com/blog/real-world-data-noaa-climate-dat...

Reverse geocoding is a task requiring quite sophisticated algorithms to implement it optimally. It's worth a dedicated blog post... You can check the source code: https://github.com/ClickHouse/ClickHouse/. See src/Dictionaries/Polygon Dictionary Implementations.h

trgn• 3 years ago

elasticsearch also has native support for vector tiles output, perfect for rendering maps in web/mobile apps. In general, any geo-app, it's not just about doing the searches/computations quickly, half the battle is efficiently delivering the data to the client.

lopkeny12ko• 3 years ago

Your critique is valid, but whatever happened to letting developers build cool technical projects for their own sake?

The author does not claim this should be used in production. As far as I can tell, this appears to be more of a novelty technical demo more than anything. Saying that the author is trying to reinvent the wheel of production-grade databases is uncharitable at best.

jackpeterfletch• 3 years ago

I actually think there’s a fairly large gap for new and innovative geospatial tools/solutions in the open-source space.

It’s true that alot of databases that have “Support for Geospatial” - but that can mean a a lot of things, often with little overlap in functionality, or very specific, inflexible access patterns that breakdown with even small variations in the problem.

I don’t think it’s too uncommon to find that none of the available solutions work for particular real world problems, and often it comes down to handcrafted solutions like this.

jjevanoorschot• 3 years ago

I've been working on a geospatial application, and one of the use cases was finding intersections between large sets of polygons. For some use cases you might not need a separate data store like this.

I used an in-memory RTree in Golang [0]. It's super fast and efficient.

[0] https://github.com/tidwall/rtree

asadawadia2• 3 years ago

is it possible to get only N closest points that match the distance predicate?

tidwall• 3 years ago

Yes. That library has a flexible kNN method built in. https://pkg.go.dev/github.com/tidwall/rtree#RTree.Nearby

ramses0• 3 years ago

https://tile38.com - Not yet had cause to use this one, but it seems like "the right one" for "geo stuff".

mrleinad• 3 years ago

Any similar .NET nuget to recommend, anyone? Thanks

Geo-Assist – An in-memory spatial engine to store and query spatial data

Related Stories