Good suggestions Jamal. I think it’s not one or the other, we need both i.e. db support
and format conversions for different engines.
We already export graph elements to edge lists, but not adjacency lists or matrices.
Will you consider submitting your code to this project?
-Kushal.
From: jamal sasha [mailto:jamalshasha@gmail.com]
Sent: Tuesday, June 24, 2014 3:36 PM
To: Datta, Kushal
Cc: graphbuilder(a)lists.01.org
Subject: Re: [GraphBuilder] Is GB project dead?
Hmm.. Well introducing db at this stage seems too early?
Maybe I am wrong, but at this point, I am still validating whether my original hypothesis
works or not.
So, most of the filter and stuff, I have already used spark to build dataset.
The issue I generally face is.. there are (say) 10 different frameworks (graphlab,
graphchi, mahout,giraph,sparkx etc)... with each having their own input requirement...
Now, I dont know which framework to use at this point of time as I still dont know which
will perform better. So, I have to try them out to see that.. But with each having their
own input requirement, I end up doing a lot of format conversions... From csv to sequence
files (for mahout) to edge lists to adjacency lists to market matrix formats..
Well, I have already gone thru the exercise.. but would be neat if graphbuilder can
atleast offer these conversions without dumping data on db?
Not sure whether I am making any sense or not?
On Wed, Jun 18, 2014 at 5:58 PM, Datta, Kushal
<kushal.datta@intel.com<mailto:kushal.datta@intel.com>> wrote:
That’s a good question. Initially, we developed Graph Builder to solve the problem of
constructing graphs from various data sources with different formats at scale. The scope
of the tool has only increased. We envision Graph Builder to evolve into a graph ETL
toolkit which will include a library of import, extract, transform and export operators
for graph. For example, the library can be used to
• Import data from a set of parquet files in HDFS
• Filter in all data related to Country Code = USA
• Calculate the join of multiple data files to correlate sales figures for every
employee
• Export the graph to a graph database such as Titan or a graph execution engine
such as GraphLab
This is the vision that we are pursuing in our group and hopefully the open source
community around Graph Builder can help us enrich it with new ideas and codebases.
-Kushal.
From: jamal sasha [mailto:jamalshasha@gmail.com<mailto:jamalshasha@gmail.com>]
Sent: Wednesday, June 18, 2014 5:36 PM
To: Datta, Kushal
Subject: Re: [GraphBuilder] Is GB project dead?
Question, will GB grow beyond just Graph construction library?
On Wed, Jun 18, 2014 at 5:30 PM, jamal sasha
<jamalshasha@gmail.com<mailto:jamalshasha@gmail.com>> wrote:
Yepp.. I guess lots of time I spend enormous time in transforming formats between
different frameworks.
Graphchi requires Market Matrix (which is not standard MM format), Graphlab requires
egelist.. Giraph has different input..
Lots of time, I just want to evaluate the performance of a framework but I spend alot of
time juggling these formats.
It would be awesome, if it can read tons of input formats and has out of the box support
for atleast these popular frameworks?
What can I expect in the next release? and when is the next release?
On Wed, Jun 18, 2014 at 5:19 PM, Datta, Kushal
<kushal.datta@intel.com<mailto:kushal.datta@intel.com>> wrote:
Hi Jamal,
The project is alive. We are making a lot of changes in the GB code internally, hence the
slowdown.
Is there anything you were particularly looking for?
Thanks,
-Kushal.
From: GraphBuilder
[mailto:graphbuilder-bounces@lists.01.org<mailto:graphbuilder-bounces@lists.01.org>]
On Behalf Of jamal sasha
Sent: Wednesday, June 18, 2014 5:08 PM
To: graphbuilder@lists.01.org<mailto:graphbuilder@lists.01.org>
Subject: [GraphBuilder] Is GB project dead?
Hi GB team,
I dont see any stories,code pushes,future features or any activity?
Is this project still going on or is it already dead?
-J
_______________________________________________
GraphBuilder mailing list
GraphBuilder@lists.01.org<mailto:GraphBuilder@lists.01.org>
https://lists.01.org/mailman/listinfo/graphbuilder
_______________________________________________
GraphBuilder mailing list
GraphBuilder@lists.01.org<mailto:GraphBuilder@lists.01.org>
https://lists.01.org/mailman/listinfo/graphbuilder