Gephi Plugin Developer Workshop

1. GephiPlugin Developer WorkshopMathieu Bastian<mathieu.bastian@gephi.org>Mountain View, CaliforniaOctober 2011

2. Welcome! Introduction to Gephi

3. Why building plug-ins?

4. Netbeans Platform

5. Gephi’s Architecture

6. Plug-ins example

7. Gephi Toolkit

8. Live codingGephiPlugin Developer Workshop

9. Introduction to GephiGephi is… A project started in 2007

10. An Exploratory Network Analysis Tool

11. An open-source platform

12. Written in Java, Swing, OpenGLGephiPlugin Developer Workshop

13. Introduction to GephiGephi has… A website, a wiki, a bug tracker, a forum and a mailing-list

14. A modular and extensible architecture

15. A codebase of ~180K lines of code

16. Long-term plans and roadmap

17. A ManifestoGephiPlugin Developer Workshop

18. Introduction to GephiGephi focus on… Performance, Usability and Modularity (and Stability!)

19. The platform, easy to use and extend

20. Networks. No diagrams or other visualizations.

21. Standards and open innovationGephiPlugin Developer Workshop

22. Introduction to GephiThe Platform’s vision Modular architecture The secret sauce for a sustainable code

23. Interoperability Use, improve or create standards

24. Integration Be exigent with UI, installation, platforms

25. Open and transparent Ideas and features come from the community

26. Public good Is and will remain open-source

27. Documentation Provide tutorials and rich JavadocGephiPlugin Developer Workshop

28. Why building plug-ins? Networks are everywhereSocial Networks, Biological Networks, Power grid, Software, Innovation… The complex world produces networks to visualize and analyze.Gephi is a generic software for all networksThe variety of networks, data sources and algorithms is huge. Gephi can’t support all of them by default so we need plug-ins.GephiPlugin Developer Workshop

29. Why building plug-ins? It’s easy to build and updatePlug-ins are easy to create and release. Development is integrated in Netbeans IDE. No JAR hell. Easy to release new version through Gephi’s website. Hack vs Plug-insPlug-ins are an incentive to build sustainable modules and share them. One can always take the source code and hack it to get things done but there is an real advantage to built on top of clean APIs.GephiPlugin Developer Workshop

30. Gephi architectureGephiPlugin Developer Workshop

31. Netbeans Platform"The NetBeans Platform is a generic framework for Swing applications. It provides the "plumbing" that, before, every developer had to write themselves”Quick StartDescription and tutorialsAPI IndexGephiPlugin Developer Workshop

32. Netbeans PlatformGephi is built on top of Netbeans Platform.

33. So, development is fully integrated in Netbeans IDE:

34. Download Gephi’ssources

35. File > Open Project in Netbeans

36. Browse existing modules or create new

37. Run Gephi. Click on Run!GephiPlugin Developer Workshop

38. Netbeans PlatformIn a few words…Netbeans Platform provides to Gephi the module system, the window system, the localization system, the preferences system and more…

39. Gephi development is integrated within Netbeans IDE.

40. A Netbeans Platform application (like Gephi) is basically a suite of modules.

41. Each module has a title, a description, a version, an author and hopefully a well defined role in the application.

42. Modules expose public APIs and depend on each other (without circular dependencies).GephiPlugin Developer Workshop

43. Gephi ArchitectureGephiPlugin Developer Workshop

44. Gephi APIs Graph API

45. Attributes API

46. Import API

47. Layout API

48. Statistics API

49. Filters API

50. Ranking API

51. Partition API

52. Project API

53. Preview API

54. Export API

55. DataLab API

56. Dynamic API

57. Generator API

58. LongTask API

59. Visualization APIRead more on the JavadocGephiPlugin Developer Workshop

60. Gephi APIs Each API has a role and a set of public packages other modules can use

61. Consult the Javadoc to see what an API does and browse classes

62. While the software evolve, APIs remain backward compatible

63. When we do changes on APIs, we wrote changes in the “API Changes” section on the Javadoc front-page

64. If you see an undocumented API, contact us or report a bugGephiPlugin Developer Workshop

65. GephiSPIsSPI vs API…API: Can be called to do something. For instance a module call the LayoutAPIto run a layout algorithm for 100 iterations.

66. SPI: Interface to be implemented. For example a plug-in implements the Layout interface to add a new layoutSPIs is the classical way how features are implemented in Gephi (and Netbeans). You’ll see some packages ending with ‘spi’ in the Javadoc. GephiPlugin Developer Workshop

67. GephiSPIs Import SPI

68. Layout SPI

69. Statistics SPI

70. Filters SPI

71. Ranking SPI

72. Generator SPI

73. DataLab SPI

74. Partition SPI

75. Preview SPI

76. Export SPI

77. Perspective SPI

78. Tools SPI

79. Workspace Persistence SPIGephiPlugin Developer Workshop

80. Graph APIHosts the Graph data. A graph is a collection of nodes and edges and could be directed, undirected or mixed. Iterate over nodes, edges or add/remove items.

81. Get node counts, degrees, neighbors…

82. Subscribe to graph events

83. Create sub-graphs (views)

84. Supports hierarchical graphs (graphs within graphs)

85. No parallel edgesGephiPlugin Developer Workshop

86. Attributes APIEach node/edge can have additional data, called attributes. Attributes can be any primitive type (Integer, Boolean, String, …), lists or dynamic. Add/Remove columns

87. Columns are identified by an id and a title.

88. Push data to nodes/edges specifying the columnGephiPlugin Developer Workshop

89. Import APIImports data from any support (files, databases. streams). Import is a two-step process: first step adds all data to a container and second step appends the container to the workspace. Import data to a new workspace or merge it to an existing

90. Verify the content of the container, fill a log/warning/error report

91. Automatically merge parallel edges

92. Doesn’t support real-time importGephiPlugin Developer Workshop

93. Layout APIProvides real-time layout algorithm execution. Layouts do nothing else than modify the X and Y coordinates of nodes. Control layout executionGephiPlugin Developer Workshop

94. Statistics APIProvides the statistics/metrics algorithm execution framework. Statistics typically use the network data to create results for each node/edge (degree, clustering coefficient, hits, …) or global. Synchronous or asynchronous algorithm execution

95. Supports progress tracking and cancellation

96. Get HTML reports from statistics execution

97. Execute dynamic statistics with sliding window supportGephiPlugin Developer Workshop

98. Filters APIProvides the framework to filter the graph and obtain a filtered sub-graph (a view). A filter can be a simple boolean predicate (age > 30) or a more complex process (k-core). Filters have parameters. Execute a filter and obtain the sub-graph

99. Create more advanced filter queries with operators (And, Or, Not…)

100. Create advanced chained filter queries

101. Export the filter result to a new workspaceGephiPlugin Developer Workshop

102. Ranking APIRanking uses numerical attribute columns to visually transform elements (nodes, edges). Typically changing the size or color of the element. Get all possible rankings for a type of element

103. Transform nodes/edges with default transformers

104. Define custom interpolations

105. Enable auto-transformation. Simply a transformation automatically applied when the graph changes.GephiPlugin Developer Workshop

106. Partition APIPartitions are sets of nodes/edges with the same value for a particular column. Get the set of values for a particular columns and the collections of items for each value.

107. Apply color transformation on the partition

108. Transform a partition into a hierarchical graphGephiPlugin Developer Workshop

109. Project APIProject API hosts the project and workspace data. Workspaces are data containers which any module can access. If data has to be shared between modules it has to lie in the Workspace. That’s exactly where each model is supposed to be stored. Create, open and close projects and workspaces

110. Retrieve and switch the current workspace

111. Subscribe to the almighty Workspace eventsGephiPlugin Developer Workshop

112. Preview APIPreview is a flexible visual rendering framework with high level of customization. The input is the graph data and a style and the output is a network map ready to be exported to PDF, SVG or PNG. Renders the graph to Processing, PDF or SVG with the same visual output

113. Customize all sort of things, from fonts to edge thickness or color

114. Get a Processing viewer out of the boxGephiPlugin Developer Workshop

115. Export APIExport’s role is to export graph or preview data to files or streams. Export graph data, for instance to a GEXF or GraphML file

116. Export preview data, typically to PDF, SVG or PNG files

117. For graph, one can choose to export the complete or only visible graphGephiPlugin Developer Workshop

118. Data Laboratory APIThe Data Laboratory works with spreadsheets and provides a large number of actions to manipulate the graph. All actions available by either right-clicking on nodes/edges or in the toolbar are accessible in the API. For instance the Search/Replace feature. Manipulate columns, Import CSV, Merge elements, clear…

119. Sophisticated merging strategies (sum, average, join…)

120. Graph manipulation (group, delete, duplicate…)

121. Search/ReplaceGephiPlugin Developer Workshop

122. Dynamic APIAPI dedicated to dynamic network states and events. Browsing dynamic networks uses the Timeline component and defines a ‘visible interval’. This API is responsible for holding and modifying that value. Retrieve/Set the current visible interval

123. Get the current time format (date, double, datetime)

124. Create DynamicGraph, a utility class to apply a sliding window on a dynamic graphGephiPlugin Developer Workshop

125. Generator APIGenerators are algorithms to create networks from a set of properties. This API simply let users execute these algorithms in a thread. Execute a generator in a thread

126. Provides progress and cancellationGephiPlugin Developer Workshop

127. LongTask APIAPI used by other modules (Statistics, Generator, Import…) to execute long tasks in separate threads with progress, error and cancel support. Execute a LongTask in a separate thread with progress, error and cancel support

128. Get informed when tasks finishes or returned errors

129. Each LongTaskExecutor only supports one task at a timeGephiPlugin Developer Workshop

130. Visualization APIStill in development. This API will eventually expose all options of the rendering engine and let users customize properties. It will also provide mouse events queues, selected elements lookup and off-screen screenshots.GephiPlugin Developer Workshop

131. Now SPIs…SPIs define how Gephi can be extended. There are simple Java interfaces like Layout, Importer, Filter or Renderer.When you implement a new SPI interface add a @ServiceProvider annotation to the class and that’s all you need to do to register your new implementation:GephiPlugin Developer Workshop

132. Import SPIExtend the range of file formats or databases supported. An importer is reading a data source and pushing data to the import container. Add new file formats

133. Add new types of databases

134. Add new spigot (API, Socket, Stream…)

135. Define custom import UI. The UI could be a single settings JPanel or a wizard.

136. How to write an Import

137. How to write a Spigot with WizardGephiPlugin Developer Workshop

138. Layout SPIAdd new layout algorithms. A layout algorithm reads the graph and modify X and Y coordinates. Add new layout algorithm

139. Define the layout properties

140. How to write a layoutGephiPlugin Developer Workshop

141. Statistics SPIAdd new statistics algorithms. Statistics write result data for each node/edge and write HTML reports. Add new statistics algorithm

142. Create a settings UI panel

143. Supports standard and dynamic statistics

144. How to write a metricGephiPlugin Developer Workshop

145. Filters SPIFilters are routines that remove nodes/edges from the graph given some parameters. Filters are built by “Builders” which act like factories. A filter builder can for instance build a filter for each attribute column. It’s called a category builder in that case. Create new filters and define parameters

146. Create a settings UI which modify the parameters

147. Create filter builders

148. How to write a filterGephiPlugin Developer Workshop

149. Ranking SPIRanking has a new SPI to create custom transformers. A transformer is transforming the node visual attributes given the list of values normalized between 0 and 1. Create new transformers

150. Create the transformer UI, a custom JPanelGephiPlugin Developer Workshop

151. Generator SPISimple SPI for adding new generators. Generator implements the LongTask interface to support progress and cancellation. Create new generators

152. Provide a settings UI

153. How to write a graph generatorGephiPlugin Developer Workshop

154. DataLab SPIThe Data Laboratory actions can be extended, whether it’s a contextual or general action. Create new node/edge context menu actions to manipulate one or multiple selected nodes/edges.

155. Create a general action that will show up in the Toolbar

156. Create a new action for manipulating attribute columns

157. Create new merge strategies

158. How to extend Data Laboratory featuresGephiPlugin Developer Workshop

159. Partition SPIThe partition SPI allows to create new transformers. However this SPI is not stable yet and will be rewritten in a near future. Create new transformers

160. Provide a settings UIGephiPlugin Developer Workshop

161. Preview SPIPreview is a highly customizable module. Item builders built items from the graph data and renderers render these items to targets (Processing, SVG and PDF). Create new item builders or replace default implementations

162. Create new renderers or replace default implementations

163. Add new properties

164. Add custom UI which will be displayed in an extra tab

165. How to write a preview rendererGephiPlugin Developer Workshop

166. Export SPIThere is three types of exports: Graph, Vector and custom. Support new graph format extension

167. Support new type of vector-based file

168. Create settings UI

169. Create new type of custom export (ex: stream)

170. How to write an exportGephiPlugin Developer Workshop

171. Perspective SPIPerspectives are a set of panels. Typically “Overview”, “Data Laboratory” and “Preview” are perspectives. Create new perspectives

172. Set which panels belong to a perspective

173. Register a perspectiveGephiPlugin Developer Workshop

174. Tools SPITools are actions users can take on the graph and are presented in the toolbar at the left of the graph window. Tools register to graph events (Mouse click, drag, press, Node click, …) and take action. Create new tools

175. Create property toolbar to configure the toolGephiPlugin Developer Workshop

176. Workspace Persistence SPIModules can implement the WorkspacePersistenceProvider interface to save/load data into .gephi project files. When the project is saved every implementation is asked to write XML data and an identifier. When the project file is opened the system find identifiers and give the XML to parse to the implementation. Add new data to the project fileGephiPlugin Developer Workshop

177. Create a Plug-in (5 minutes)What a plug-in really is…A plug-in is nothing else than a Netbeans module packaged in a single file with a NBM extension. It’s basically a JAR with additional meta-data.There is no differences between Gephi’s core modules and plug-ins. There are all Netbeans modules. In a sense Gephi is just a suite of plug-ins which work together and depend on each other.To create a new module…Once Gephi project is open, right click on “Modules” > “Add new…”GephiPlugin Developer Workshop

178. Create a Plug-in (5 minutes)Right click on your module > Properties… Manage dependenciesDependencies are explicit between modules. You control which other modules your module depends on. You also control which packages are publicly visible for other modules.Your module will use APIs set by other modules and/or define its own APIs which others can use.Depends onMy ModuleGephiPlugin Developer Workshop

179. Create a Plug-in (5 minutes)Right click in your module > Properties… Module brandingEach module has a name, a description, a license, a version and an author.When you update your module, increase the specification version number.GephiPlugin Developer Workshop

180. Create a Plug-in (5 minutes)Right click on your module > Create NBM…It creates a single file you can distribute and share.Gephi’s website let you upload your NBM file out of the box. You can also update your plug-in with a new version.For multi-modules plug-ins, it also accepts a zip package of several NBMs.GephiPlugin Developer Workshop

181. Plugin examples – Circular LayoutAdds a new Layout. Standard use of the Layout SPI.GephiPlugin Developer Workshop

182. Plugin examples – Complex GeneratorsAdds new Generators with UI. Standard use of the Generator SPI.GephiPlugin Developer Workshop

183. Plugin examples – Social Network AnalysisAdds new statistics. Standard use of the Statistics SPI.GephiPlugin Developer Workshop

184. Plugin examples – LinkfluencePluginAdds new context and global actions in the Data Laboratory. The plug-in has been developed by the Linkfluence company and open-sourced.GephiPlugin Developer Workshop

185. Plugin examples – HTTP GraphStreams the HTTP graph while the user is browsing the web. Implemented as a Generator. It’s a non standard use of the Generator SPI but it works.GephiPlugin Developer Workshop

186. Plugin examples – Script consoleAdds a Python console as a new component. Doesn’t use a SPI but the plugin uses various APIs.GephiPlugin Developer Workshop

187. Plugin examples – SeadragonpluginExports the graph from Preview to a zoomable web interface. Extends the Export SPI and uses the Preview API.GephiPlugin Developer Workshop

188. Plugin distribution and update Plug-ins are available and can be published at http://gephi.org/plugins

189. Subscribe to the gephi-plugins mailing-list.

190. Plug-ins are also available directly from Gephi’sPlugin Center (Tools > Plugins).

191. Like any other module, plug-ins are updated when a new version is available.GephiPlugin Developer Workshop

192. Gephi ToolkitThe Gephi Toolkit packages essential modules in a standard Java library. The toolkit is just a single JAR that anyone could reuse in new Java applications, for example: Headless Gephi from an input graph file (DOT, GML, GEXF, …) to the resulting PDF, with a list of settings.

193. Servlet Use Gephi toolkit to create graph snippets PNG images automatically in a servlet.The Toolkit is release separately from Gephi and can be download at http:/gephi.org/toolkitGephiPlugin Developer Workshop

194. Gephi Toolkit The Toolkit is basically Gephi without the user interface modules

195. One can find documentation and tutorials on the wiki: Toolkit Portal

196. It has a separate Javadoc, as SPI’s implementations are also included

197. Plug-ins can also be used in the Toolkit. Consult the wiki

198. It’s easy to use from JRuby or JythonGephiPlugin Developer Workshop

199. Thank you!Questions?GephiPlugin Developer Workshop

200. Book recommendationsAlso checkout How to Design a Good API and Why it Matters (Joshua Bloch)GephiPlugin Developer Workshop

Gephi Plugin Developer Workshop

More Related Content

What's hot (20)

Similar to Gephi Plugin Developer Workshop (20)

Recently uploaded (20)

Gephi Plugin Developer Workshop