1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
|
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Serial communication library for Akka and Scala.">
<link rel="shortcut icon" type="image/png" href="https://jodersky.github.io/flow/assets/images/logo.png"/>
<title>flow - Developer Guide</title>
<link rel="stylesheet" href="https://jodersky.github.io/flow/assets/lib/bootstrap/css/bootstrap.min.css">
<link rel="stylesheet" href="https://jodersky.github.io/flow/assets/lib/fontawesome/css/font-awesome.min.css">
<link rel="stylesheet" href="https://jodersky.github.io/flow/assets/stylesheets/solarized-dark.css">
<link rel="stylesheet" href="https://jodersky.github.io/flow/assets/stylesheets/main.css">
<script src="https://jodersky.github.io/flow/assets/lib/jquery/jquery-1.11.3.min.js"></script>
</head>
<body>
<header>
<nav class="navbar navbar-static-top navbar-default">
<div class="container">
<div class="navbar-header">
<a class="navbar-brand" href="https://jodersky.github.io/flow">
<img src="https://jodersky.github.io/flow/assets/images/logo.png" alt="flow logo">
flow
</a>
</div>
<ul class="nav navbar-nav navbar-right">
<li><a href="https://jodersky.github.io/flow/documentation">Documentation & Help</a></li>
<li><a href="https://jodersky.github.io/flow/downloads">Download</a></li>
<li><a href="https://github.com/jodersky/flow">GitHub</a></li>
</ul>
</div>
</nav>
</header>
<div class="container">
<h1 id="building-from-source">Building from Source</h1>
<p>A complete build of flow involves two parts</p>
<ol>
<li>
<p>Building Scala sources (the front-end), resulting in a platform independent artifact (i.e. a jar file).</p>
</li>
<li>
<p>Building C sources (the back-end), yielding a native library that may only be used on systems resembling the platform for which it was compiled.</p>
</li>
</ol>
<p>Both steps are independent, their only interaction being a header file generated by the JDK utility <code class="highlighter-rouge">javah</code> (see <code class="highlighter-rouge">sbt javah</code> for details), and may therefore be built in any order.</p>
<h2 id="building-scala-sources">Building Scala Sources</h2>
<p>Run <code class="highlighter-rouge">sbt flow-main/packageBin</code> in the base directory. This simply compiles Scala sources as with any standard sbt project and packages the resulting class files in a jar.</p>
<h2 id="building-native-sources">Building Native Sources</h2>
<p>The back-end is managed by GNU Autotools and all relevant files are contained in <code class="highlighter-rouge">flow-native</code>.</p>
<aside class="notice">
<h3 id="aside-autotools-introduction">Aside: Autotools Introduction</h3>
<p>Autotools is a suite of programs constituting a sort of “meta-build system”. It is used to generate a platform-independent build script known as <code class="highlighter-rouge">./configure</code>, which, when run, will analyze the current system (search for a C compiler, required libraries etc) and produce a <code class="highlighter-rouge">Makefile</code>. The makefile in turn is system-specific and can be used to create the final binary. In summary the build process is as follows:</p>
<ol>
<li>Autotools (specifically the program <code class="highlighter-rouge">autoreconf</code>) generates <code class="highlighter-rouge">./configure</code>, this happens on the developer’s machine</li>
<li><code class="highlighter-rouge">./configure</code> is run on the host computer</li>
<li><code class="highlighter-rouge">make</code> is run to produce a binary, also on the host computer</li>
</ol>
<p>In a typical, source-controlled repository, only a bootstrapping script that calls Autotools is checked into version control. However, source <em>releases</em> include the generated <code class="highlighter-rouge">./configure</code> script. An end-user then downloads a source release and only has to run <code class="highlighter-rouge">./configure && make</code>.</p>
<p>However, since flow does currently not provide source releases (not to be confused with source repository or Git tags), the developer’s machine is the same as the host machine and so the bootstrapping process always needs to be performed.</p>
</aside>
<h3 id="build-process">Build Process</h3>
<p>Several steps are involved in producing the native library:</p>
<ol>
<li>
<p>Bootstrap the build (run this once, if <code class="highlighter-rouge">./configure</code> does not exist).</p>
<ol>
<li>Check availability of dependencies: autotools and libtool (on Debian-based systems run <code class="highlighter-rouge">apt-get install build-essential autoconf automake libtool</code>)</li>
<li>Run <code class="highlighter-rouge">./bootstrap</code></li>
</ol>
</li>
<li>
<p>Compile</p>
<ol>
<li>Check availability of dependencies: C compiler and JDK (1.8 or above)</li>
<li>Run <code class="highlighter-rouge">./configure && make</code>.
<em>Note: should you encounter an error about a missing “jni.h” file, try setting the JAVA_HOME environment variable to point to base path of your JDK installation.</em></li>
</ol>
</li>
<li>
<p>Install</p>
<p>The native library is now ready and can be:</p>
<ul>
<li>
<p>copied to a local directory: <code class="highlighter-rouge">DESTDIR=$(pwd)/<directory> make install</code></p>
</li>
<li>
<p>installed system-wide: <code class="highlighter-rouge">make install</code></p>
</li>
<li>
<p>put into a “fat” jar, useful for dependency management with SBT (see next section)</p>
</li>
</ul>
</li>
</ol>
<h3 id="creating-a-fat-jar">Creating a Fat Jar</h3>
<p>The native library produced in the previous step may be bundled into a “fat” jar so that it can be included in SBT projects through its regular dependency mechanisms. In this process, sbt basically acts as a wrapper script around Autotools, calling the native build process and packaging generated libraries. Running <code class="highlighter-rouge">sbt flow-native/packageBin</code> in the base directory produces the fat jar in <code class="highlighter-rouge">flow-native/target</code>.</p>
<p>Note: an important feature of fat jars is to include native libraries for several platforms. To copy binaries compiled on other platforms to the fat jar, place them in a subfolder of <code class="highlighter-rouge">flow-native/lib_native</code>. The subfolder should have the name <code class="highlighter-rouge">$(arch)-$(kernel)</code>, where <code class="highlighter-rouge">arch</code> and <code class="highlighter-rouge">kernel</code> are, respectively, the lower-case values returned by <code class="highlighter-rouge">uname -m</code> and <code class="highlighter-rouge">uname -s</code>.</p>
<h3 id="note-about-versioning">Note About Versioning</h3>
<p>The project and package versions follow a <a href="http://semver.org/">semantic</a> pattern: <code class="highlighter-rouge">M.m.p</code>, where</p>
<ul>
<li>
<p><code class="highlighter-rouge">M</code> is the major version, representing backwards incompatible changes</p>
</li>
<li>
<p><code class="highlighter-rouge">m</code> is the minor version, indicating backwards compatible changes such as new feature additions</p>
</li>
<li>
<p><code class="highlighter-rouge">p</code> is the patch number, representing internal modifications such as bug-fixes</p>
</li>
</ul>
<p>Usually (following most Linux distribution’s conventions), shared libraries produced by a project <code class="highlighter-rouge">name</code> of version <code class="highlighter-rouge">M.m.p</code> are named <code class="highlighter-rouge">libname.so.M.m.p</code>. However, since when accessing shared libraries through the JVM, only the <code class="highlighter-rouge">name</code> can be specified and no particular version, the convention adopted by flow is to append <code class="highlighter-rouge">M</code> to the library name and always keep the major version at zero. E.g. <code class="highlighter-rouge">libflow.so.3.1.2</code> becomes <code class="highlighter-rouge">libflow3.so.0.1.2</code>.</p>
<h1 id="publishing-and-releasing">Publishing and Releasing</h1>
<p>The release process managed with the <code class="highlighter-rouge">sbt-release</code> plugin. See ‘project/Release.scala’ for a description of the various steps involved.</p>
<p>Here are some important notes on creating a release:</p>
<ul>
<li>
<p>During a release, only readily available libraries in <code class="highlighter-rouge">lib_native</code> are packaged into the fat jar, no local compilation is performed. The rationale behind this is that while native libraries rarely change, they are tied to the version of libc of the compiling system. Since the releases are mostly done on a cutting-edge OS, compiling native libraries locally could break compatibility with older systems.</p>
</li>
<li>
<p>Currently, the release script does not handle uploading the native libraries archive (don’t confuse this with the fat jar, which is uploaded). If creating a release that changed the native libraries or added support for more platforms, creating and uploading a new native archive must be done manually.</p>
</li>
<li>
<p>Don’t forget to update the website after creating a new release.</p>
</li>
</ul>
</div>
<script src="https://jodersky.github.io/flow/assets/lib/bootstrap/js/bootstrap.min.js"></script>
</body>
</html>
|
