aboutsummaryrefslogtreecommitdiff
path: root/Documentation/developer.md
blob: 3f74890f294fc5a541d02773a54a570871945bbe (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
---
layout: page
title: Developer Guide
---
# Content
* TOC
{:toc}

# Building from Source
A complete build of flow involves two parts

1. Building Scala sources (the front-end), resulting in a platform independent artifact (i.e. a jar file).

2. Building C sources (the back-end), yielding a native library that may only be used on systems resembling the platform for which it was compiled.

Both steps are independent, their only interaction being a header file generated by the JDK utility `javah` (see `sbt javah` for details), and may therefore be built in any order.

## Building Scala Sources
Run `sbt core/package` in the base directory. This simply compiles Scala sources as with any standard sbt project and packages the resulting class files into a jar.

## Building Native Sources
The back-end is managed by CMake and all relevant files are contained in `flow-native/src`.

### Build Process
Several steps are involved in producing the native library:

1. Bootstrap the build (run this once, if `Makefile` does not exist).

	1. Required dependencies: CMake (2.8 or higher), JDK (1.8 or above)
    2. Run `cmake .`

2. Compile

    1. Run `make`.
       *Note: should you encounter an error about a missing "jni.h" file, try setting the JAVA_HOME environment variable to point to the base path of your JDK installation.*

3. Install

    The native library is now ready and can be:

	- copied to a local directory: `DESTDIR=$(pwd)/<directory> make install`

    - installed system-wide: `make install`

    - put into a "fat" jar, useful for dependency management with sbt (see next section)

### Creating a Fat Jar
The native library produced in the previous step may be bundled into a "fat" jar so that it can be included in SBT projects through its regular dependency mechanisms. In this process, sbt basically acts as a wrapper script around CMake, calling the native build process and packaging generated libraries. Running `sbt native/package` produces the fat jar in `flow-native/target`.

Note: an important feature of fat jars is to include native libraries for several platforms. To copy binaries compiled on other platforms to the fat jar, place them in a subfolder of `flow-native/lib_native`. The subfolder should have the name `$(arch)-$(kernel)`, where `arch` and `kernel` are, respectively, the lower-case values returned by `uname -m` and `uname -s`.

### Note About Versioning
The project and package versions follow a [semantic](http://semver.org/) pattern: `M.m.p`, where

- `M` is the major version, representing backwards incompatible changes to the public API

- `m` is the minor version, indicating backwards compatible changes such as new feature additions

- `p` is the patch number, representing internal modifications such as bug-fixes

Usually (following most Linux distribution's conventions), shared libraries produced by a project `name` of version `M.m.p` are named `libname.so.M.m.p`. However, since when accessing shared libraries through the JVM, only the `name` can be specified and no particular version, the convention adopted by flow is to append `M` to the library name and always keep the major version at zero. E.g. `libflow.so.3.1.2` becomes `libflow3.so.0.1.2`.

# Testing

`flow-samples` directory contains fully functional application examples of flow. To run an example, change to the base directory of flow and run sbt samples<SampleName>/run.
All projects, including samples, can be listed by running sbt projects.

To be able connect You can use real device (arduino) burned with sample-echo (`dev/arduino-terminal`) code, or create Virtual Serial Port pair

[socat (SOcket CAT)](http://www.dest-unreach.org/socat/) – multipurpose relay – is a command line based utility that establishes two bidirectional byte streams and transfers data between them.
socat is #4 on the Top 100 Network Security Tools list, available in most distro repositories (on Debian/Ubuntu sudo apt-get install socat does the trick), really light on resources, and very efficient.

To create a pair of VSP’s
```socat -d -d pty,raw,echo=0 pty,raw,echo=0```

you will get something like
```
socat[5894] N PTY is /dev/ttys002
socat[5894] N PTY is /dev/ttys003
socat[5894] N starting data transfer loop with FDs [5,5] and [7,7]
```
and that’s it! As long as the socat is running, you have a pair of VSP’s open (their names are printed by socat on initialization). See [socat man page](http://www.dest-unreach.org/socat/doc/socat.html) for more details on what the above command does.
Now You can connect to first socket ( in this case `/dev/ttys002`) using some sample (or Your code), and use second for monitoring or/and sending messages 
To send - use command 
```
echo 'Hello World' > /dev/ttys003
```
To listen - use command
```
cat < /dev/ttys003
```

Connecting executable and VSP
```
socat -d -d pty,raw,echo=0 "exec:myprog ...,pty,raw,echo=0"
```
where the executable `myprog` will be connected with the VSP through stdio.

For example Echo-server would look like 
```
socat -d -d pty,raw,echo=0 "exec:/bin/cat,pty,raw,echo=0"
```

# Publishing and Releasing
The release process is managed with the `sbt-release` plugin. See 'project/Release.scala' for a description of the various steps involved.

Here are some important notes on creating a release:

- During a release, only readily available libraries in `lib_native` are packaged into the fat jar, no local native compilation is performed. The rationale behind this is that while native libraries rarely change, they are still tied to the version of libc of the compiling system. Since the releases are mostly done on a development machine (cutting-edge OS), compiling native libraries locally could break compatibility with older systems.

- Currently, the release script does not handle uploading the native libraries archive (don't confuse this with the fat jar, which is uploaded). If creating a release that changed the native libraries or added support for more platforms, creating and uploading a new native archive must be done manually.

- Don't forget to update the website after creating a new release:

    - Run `sbt makeSite` to generate documentation in `target/site/`
	- Checkout GitHub Pages branch `git checkout gh-pages`
	- Copy contents of `target/site/` to `documentation/M.m/`
	- Update `_config.yml` with latest version
	- Push to GitHub