Neo4j graph database - any experiences?
-
Would a standard Document DB e.g. Mongo not fit the bill? ie. each document is the node and contained within would be an array of connections.
Dave Find Me On:Web|Facebook|Twitter|LinkedIn Folding Stats: Team CodeProject
DaveAuld wrote:
Would a standard Document DB e.g. Mongo not fit the bill?
I've pretty much dismissed NoSQL DB's because documents are not relational. Yes, you can put "reference id's" into them, but this requires that the client resolve the reference, which results in additional round trips to the server. What I want to do is query the semantic structure at any level (including the abstract levels) and have it return the properties of the entire structure. Additionally, I'd like to be able to create queries that represent a join -- two or more structures that share common sub-structures. Again, this would require lots of round trips in a NoSQL DB, whereas I'd much rather like the query to be resolved by the DB itself. I have a working prototype of what I want implemented on top of a SQL database, generating the joins through schema inspection, but it seems like this is what a graph database should be able to do more elegantly (it would still require generating the query of course), so I'm concerned about not just re-inventing the wheel, but that there are better and more optimized solutions out there already that I could use. Regardless, whichever way the wind blows, this is the next article I'm working on. :) Marc
Imperative to Functional Programming Succinctly Higher Order Programming
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
What kind of graph, and what kind of queries? Why are semantics involved in this? You made it sound complicated, except
Marc Clifton wrote:
this manifests as a node with no properties, only connections
Ok that sounds pretty bare-bones, my favourite kind of graph, no-nonsense, just the graph. Here's a completely off-the-charts nuts idea (or at least that's what people keep telling me about this) - roll your own. There's no learning curve, there's no "fighting with stupid configs", no mismatch between what you want and what it does, it supports all the features you want because you put them in there. Win all around. Should also be interesting to read about, or maybe even re-use.
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
I've never heard of such a monster. You may end up creating this engine yourself. Nothing like being an inventor and starting a company around this. Make some cash! ;)
A guide to posting questions on CodeProject
Click this: Asking questions is a skill. Seriously, do it.
Dave Kreskowiak -
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
-
What kind of graph, and what kind of queries? Why are semantics involved in this? You made it sound complicated, except
Marc Clifton wrote:
this manifests as a node with no properties, only connections
Ok that sounds pretty bare-bones, my favourite kind of graph, no-nonsense, just the graph. Here's a completely off-the-charts nuts idea (or at least that's what people keep telling me about this) - roll your own. There's no learning curve, there's no "fighting with stupid configs", no mismatch between what you want and what it does, it supports all the features you want because you put them in there. Win all around. Should also be interesting to read about, or maybe even re-use.
harold aptroot wrote:
roll your own.
I'm essentially doing that right now, on top of a relational database. However, if I can discern a real benefit (or issue) with a graph database, that would be good to know. It seems possible to do with Neo4j, I just need to take a couple hours and play with it.
harold aptroot wrote:
There's no learning curve, there's no "fighting with stupid configs", no mismatch between what you want and what it does, it supports all the features you want because you put them in there. Win all around. Should also be interesting to read about, or maybe even re-use.
The funny thing is, what I'm doing could be done with a traditional RDBMS as long as it has good FK constraints built into the schema. And while there is no learning curve with "roll your own", there can be one heck of a development/testing curve! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
-
kmoorevs wrote:
XML and LINQ maybe?
Hmmm, not really what I have in mind. That would end up looking a lot like a NoSQL database then. Marc
Imperative to Functional Programming Succinctly Higher Order Programming
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
If I remember my math correctly, you can have a graph that has null edge, but not null vertice. So if neo4j is done correctly, you can't do it. Depending on your definition of correctly. And as for there not being any semantic databases, did you really search? Maybe I'm misunderstanding your perception of semantic database, but there seems to be several out there. Amongst others, Oracle supports semantic modeling (and graphs and ontologies) as features of Oracle spatial[^], if you can afford it. :rolleyes: BTW there's a .Net version called Neo4net if you don't want to get dirty with Java.
Wrong is evil and must be defeated. - Jeff Ello[^]
-
harold aptroot wrote:
roll your own.
I'm essentially doing that right now, on top of a relational database. However, if I can discern a real benefit (or issue) with a graph database, that would be good to know. It seems possible to do with Neo4j, I just need to take a couple hours and play with it.
harold aptroot wrote:
There's no learning curve, there's no "fighting with stupid configs", no mismatch between what you want and what it does, it supports all the features you want because you put them in there. Win all around. Should also be interesting to read about, or maybe even re-use.
The funny thing is, what I'm doing could be done with a traditional RDBMS as long as it has good FK constraints built into the schema. And while there is no learning curve with "roll your own", there can be one heck of a development/testing curve! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
Have a look at OQGRAPH Engine for MariaDB[^] From what you're writing this sounds like what you're looking for: The Open Query GRAPH computation engine, or OQGRAPH as the engine itself is called, allows you to handle hierarchies (tree structures) but also complex graphs (nodes having many connections in several directions). [^]
Espen Harlinn Chief Architect - Powel AS Projects promoting programming in "natural language" are intrinsically doomed to fail. Edsger W.Dijkstra
-
Have a look at OQGRAPH Engine for MariaDB[^] From what you're writing this sounds like what you're looking for: The Open Query GRAPH computation engine, or OQGRAPH as the engine itself is called, allows you to handle hierarchies (tree structures) but also complex graphs (nodes having many connections in several directions). [^]
Espen Harlinn Chief Architect - Powel AS Projects promoting programming in "natural language" are intrinsically doomed to fail. Edsger W.Dijkstra
Espen Harlinn wrote:
Have a look at OQGRAPH Engine for MariaDB[^]
Wow, that does look very very promising. Much appreciated! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
-
If I remember my math correctly, you can have a graph that has null edge, but not null vertice. So if neo4j is done correctly, you can't do it. Depending on your definition of correctly. And as for there not being any semantic databases, did you really search? Maybe I'm misunderstanding your perception of semantic database, but there seems to be several out there. Amongst others, Oracle supports semantic modeling (and graphs and ontologies) as features of Oracle spatial[^], if you can afford it. :rolleyes: BTW there's a .Net version called Neo4net if you don't want to get dirty with Java.
Wrong is evil and must be defeated. - Jeff Ello[^]
Jörgen Andersson wrote:
but not null vertice.
I would still have vertices, it's just that some nodes are abstract (like abstract classes) and require a "is a kind of" relationship to another node, perhaps abstract, perhaps with some concrete properties.
Jörgen Andersson wrote:
as features of Oracle spatial[^], if you can afford it.
I'm staying away from RDF and OWL as it's silly complex for structures that are more (and different) than just subject-predicate-object triples. This actually excludes a lot of graph databases, as about 50% of them seem to be tailored to working with RDF triples (there's a long long list on wikipedia, I'm of course also looking for something that hopefully works painlessly with .NET.) Marc
Imperative to Functional Programming Succinctly Higher Order Programming
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
Did you look at orientdb... it seems to be a mix between a document db and a graph db and a relational db
-
Did you look at orientdb... it seems to be a mix between a document db and a graph db and a relational db
Member 3752561 wrote:
Did you look at orientdb... it seems to be a mix between a document db and a graph db and a relational db
No, I hadn't. Thanks for providing the link. I'm impressed with their clean documentation and simple SQL-like query language. Much easier to understand than Neo4j's syntax! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
I have used neo4j in a couple large projects and absolutely love it. I still use mysql for transaction data (i.e. accounting, it's just more practical), and mongodb for centralizing document stores (images, uploads, css/js files), but for what you described neo would work just fine. Keep in mind you can also apply labels to the nodes to allow you to query the data in a more meaningful way. Neo4j isn't the end all/be all, but it's very useful for me, not to mention because of it's nature of related nodes, super quick for the data extraction that I do.
-
Member 3752561 wrote:
Did you look at orientdb... it seems to be a mix between a document db and a graph db and a relational db
No, I hadn't. Thanks for providing the link. I'm impressed with their clean documentation and simple SQL-like query language. Much easier to understand than Neo4j's syntax! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
i think the biggest problem for this style of project (orientdb, neo4j, mongodb etc) is the fact that client are develop and maintain by external developper (now mongodb have internal dev for each port of client and that s a very cool thing) if you look at the binary client for orientdb in .net, the last release is 2 years ago .... I think if you choose oriendb , you must do your own client.
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
I've been looking at Neo4J recently. I'm not quite sure what you want to achieve but if it helps, individual nodes and relationships can have whatever properties you want or none at all (the equivalent of properties in a sql database would be fields that are part of the hard-coded structure of the table and so are fixed even if they are optional). You can group nodes by assigning labels to them eg "Person". Nodes that have the same label don't have to share the same properties and a node can have as many properties as you like within practical limits. Labels help you to search for nodes or relationships that have something in common. I generally like this kind of tagging because you can use tags to group different objects in different ways which is more fluid than trying to impose a structure that isn't supposed to change. If you download the Neo4J community edition, it comes with a built-in web server that enables you to try out queries in your browser and see a visual representation of the graph
-
Espen Harlinn wrote:
Have a look at OQGRAPH Engine for MariaDB[^]
Wow, that does look very very promising. Much appreciated! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
If you try it out, I'd like to know what you think about it. I'm probably going to take a close look at it, but it's always nice to have a second opinion ;-)
Espen Harlinn Chief Architect - Powel AS Projects promoting programming in "natural language" are intrinsically doomed to fail. Edsger W.Dijkstra
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
I do not know if I understand what you want, but how do you use such a database. If the DB engine do not use concept of internal structure of the node, it cannot select/filter a concrete node, unless you already selected it. So the DB engine could do only one type of operation - load all database in memory. No indexes could be build without inner structure of nodes in the graph. To achieve performance for your model I suggest ArangoDB. In a graph view, you could achieve simple graph by just adding "empty documents" into the database with the overflow of a single hash (ID) for every node and two hashes ("_from" and "_to") for every edge. As for Neo4j I'm almost sure it can be used with nodes/edges without properties.
-
I've been looking at graph database technology and was wondering if anyone here had any concrete experience with Neo4j[^]. A specific question that I have is that it seems that graph databases (though maybe it's specific to Neo4j) are "fact" oriented -- in other words, nodes are typically concrete facts. What I'm looking for is a graph database that supports abstract concepts, and this manifests as a node with no properties, only connections. Why do I want this? Because I am looking at how to persist the structure (semantics) of the data as well the data itself. As far as I can tell, there is no such thing as a "semantic database" - basically, a merging of a relational database, semantics, and graphs. Thanks in advance! Marc
Imperative to Functional Programming Succinctly Higher Order Programming
What you need is a mature, full-featured, hi-performance object database management system ("ODBMS"). Most of the No-SQL options are missing features for what you want to accomplish. SQL databases are a poor solution for modeling graph data structures (or any highly complex db schema). Checkout the top performing database in this comparision of leading database systems: http://velocitydb.com/Compare.aspx[^] The features of VelocityDb are available in stand-alone or server versions: http://velocitydb.com/Features.aspx No-SQL databases just don't have such rich feature sets. And the open source VelocityGraph add-on link is at top of the home page: www.VelocityDb.com I've been using VelocityDb after many years with Versant and after looking at heavy-weight ODBMS systems such as Gemstone. I'm very happy with VelocityDb. My class hierarchy IS the database schema. Update your classes in your code, you've just updated your db schema.
-
What you need is a mature, full-featured, hi-performance object database management system ("ODBMS"). Most of the No-SQL options are missing features for what you want to accomplish. SQL databases are a poor solution for modeling graph data structures (or any highly complex db schema). Checkout the top performing database in this comparision of leading database systems: http://velocitydb.com/Compare.aspx[^] The features of VelocityDb are available in stand-alone or server versions: http://velocitydb.com/Features.aspx No-SQL databases just don't have such rich feature sets. And the open source VelocityGraph add-on link is at top of the home page: www.VelocityDb.com I've been using VelocityDb after many years with Versant and after looking at heavy-weight ODBMS systems such as Gemstone. I'm very happy with VelocityDb. My class hierarchy IS the database schema. Update your classes in your code, you've just updated your db schema.
licence per developper : 400$ and Deployment licenses/royalty "Our runtime royalty is determined on a case by case basis. We would like you to suggest a pricing model that fits your business. It might be a share of your profits, a fixed amount per deployment or something else. We are very flexible but we do need revenue to maintain development, support and pay our staff." so not a fixed price.... we don t know how much we will pay when we'll go in production.... licence per developper is not so high (if the performance are here) and we can use it for free for trial usage for less than 3 user but doesn't know how much we will pay or how much we will need to tax our customer to install the db is unacceptable, specialy if we make a startup or little business...
-
licence per developper : 400$ and Deployment licenses/royalty "Our runtime royalty is determined on a case by case basis. We would like you to suggest a pricing model that fits your business. It might be a share of your profits, a fixed amount per deployment or something else. We are very flexible but we do need revenue to maintain development, support and pay our staff." so not a fixed price.... we don t know how much we will pay when we'll go in production.... licence per developper is not so high (if the performance are here) and we can use it for free for trial usage for less than 3 user but doesn't know how much we will pay or how much we will need to tax our customer to install the db is unacceptable, specialy if we make a startup or little business...
The database engine is the foundation of an entire software project. Best your select the right horse to ride at the outset. Developer time savings, code re-use and saving objects or graph-nodes with simple code (no transforming your objects or nodes into rows-and-columns or a structure required by the persistence engine) pay for the minimal license costs. If you checked-out the performance comparisons, you'll realize you can get away with a fraction of the hardware / server costs for this kind of database. Check how VelocityDb on a $1,000 Costco PC compares against $1 million+ hardware with Oracle. It may be worth your while to get a price quote for your specific business case and revenue model, so you can make an informed decision. I've been in software since 1981 and have reviewed many, many data persistence schemes. This one has saved me time, money and mis-steps. But, I don't know the specifics of your project.