Squash changes for protoc-artifacts support into a single commit, so it

can be easily cherry-picked to already released tags for publishing
released protoc

Change-Id: I3aa1d2047564b12e457dbb1c26560d56b872f68a

3 files changed, 476 insertions, 0 deletions

diff --git a/protoc-artifacts/README.md b/protoc-artifacts/README.md
new file mode 100644
index 00000000..3a530197
--- /dev/null
+++ b/protoc-artifacts/README.md
@@ -0,0 +1,120 @@
+# Build scripts that publish pre-compiled protoc artifacts
+``protoc`` is the compiler for ``.proto`` files. It generates language bindings
+for the messages and/or RPC services from ``.proto`` files.
+
+Because ``protoc`` is a native executable, the scripts under this directory
+build and publish a ``protoc`` executable (a.k.a. artifact) to Maven
+repositories. The artifact can be used by build automation tools so that users
+would not need to compile and install ``protoc`` for their systems.
+
+## Versioning
+The version of the ``protoc`` artifact must be the same as the version of the
+Protobuf project.
+
+## Artifact name
+The name of a published ``protoc`` artifact is in the following format:
+``protoc-<version>-<os>-<arch>.exe``, e.g., ``protoc-3.0.0-alpha-3-windows-x86_64.exe``.
+
+## System requirement
+Install [Apache Maven](http://maven.apache.org/) if you don't have it.
+
+The scripts only work under Unix-like environments, e.g., Linux, MacOSX, and
+Cygwin or MinGW for Windows. Please see ``README.md`` of the Protobuf project
+for how to set up the build environment.
+
+## To install artifacts locally
+The following command will install the ``protoc`` artifact to your local Maven repository.
+```
+$ mvn install
+```
+
+## Cross-compilation
+The Maven script will try to detect the OS and the architecture from Java
+system properties. It's possible to build a protoc binary for an architecture
+that is different from what Java has detected, as long as you have the proper
+compilers installed.
+
+You can override the Maven properties ``os.detected.name`` and
+``os.detected.arch`` to force the script to generate binaries for a specific OS
+and/or architecture. Valid values are defined as the return values of
+``normalizeOs()`` and ``normalizeArch()`` of ``Detector`` from
+[os-maven-plugin](https://github.com/trustin/os-maven-plugin/blob/master/src/main/java/kr/motd/maven/os/Detector.java).
+Frequently used values are:
+- ``os.detected.name``: ``linux``, ``osx``, ``windows``.
+- ``os.detected.arch``: ``x86_32``, ``x86_64``
+
+For example, MingGW32 only ships with 32-bit compilers, but you can still build
+32-bit protoc under 64-bit Windows, with the following command:
+```
+$ mvn install -Dos.detected.arch=x86_32
+```
+
+## To push artifacts to Maven Central
+Before you can upload artifacts to Maven Central repository, make sure you have
+read [this page](http://central.sonatype.org/pages/apache-maven.html) on how to
+configure GPG and Sonatype account.
+
+You need to perform the deployment for every platform that you want to
+suppport. DO NOT close the staging repository until you have done the
+deployment for all platforms. Currently the following platforms are supported:
+- Linux (x86_32 and x86_64)
+- Windows (x86_32 and x86_64) with
+ - Cygwin with MinGW compilers (both x86_32 and x86_64)
+ - MSYS with MinGW32 (x86_32 only)
+- MacOSX (x86_32 and x86_64)
+
+Remove any ``SNAPSHOT`` or ``pre`` suffix from the version string before
+deploying.
+
+Use the following command to deploy artifacts for the host platform to a
+staging repository.
+```
+$ mvn clean deploy -P release
+```
+It creates a new staging repository. Go to
+https://oss.sonatype.org/#stagingRepositories and find the repository, usually
+in the name like ``comgoogle-123``.
+
+You will want to run this command on a different platform. Remember, in
+subsequent deployments you will need to provide the repository name that you
+have found in the first deployment so that all artifacts go to the same
+repository:
+```
+$ mvn clean deploy -P release -Dstaging.repository=comgoogle-123
+```
+
+A 32-bit artifact can be deployed from a 64-bit host with
+``-Dos.detected.arch=x86_32``
+
+When you have done deployment for all platforms, go to
+https://oss.sonatype.org/#stagingRepositories, verify that the staging
+repository has all the binaries, close and release this repository.
+
+### Tips for deploying on Windows
+Under Windows the following error may occur: ``gpg: cannot open tty `no tty':
+No such file or directory``. This can be fixed by configuring gpg through an
+active profile in ``.m2\settings.xml`` where also the Sonatype password is
+stored:
+```xml
+<settings>
+  <servers>
+    <server>
+      <id>ossrh</id>
+      <username>[username]</username>
+      <password>[password]</password>
+    </server>
+  </servers>
+  <profiles>
+    <profile>
+      <id>gpg</id>
+      <properties>
+        <gpg.executable>gpg</gpg.executable>
+        <gpg.passphrase>[password]</gpg.passphrase>
+      </properties>
+    </profile>
+  </profiles>
+  <activeProfiles>
+    <activeProfile>gpg</activeProfile>
+  </activeProfiles>
+</settings>
+```
diff --git a/protoc-artifacts/build-protoc.sh b/protoc-artifacts/build-protoc.sh
new file mode 100755
index 00000000..96ca97c2
--- /dev/null
+++ b/protoc-artifacts/build-protoc.sh
@@ -0,0 +1,222 @@
+#!/bin/bash
+
+# Builds protoc executable into target/protoc.exe
+# To be run from Maven.
+# Usage: build-protoc.sh <OS> <ARCH>
+# <OS> and <ARCH> are ${os.detected.name} and ${os.detected.arch} from os-maven-plugin
+OS=$1
+ARCH=$2
+
+if [[ $# < 2 ]]; then
+  echo "No arguments provided. This script is intended to be run from Maven."
+  exit 1
+fi
+
+# Under Cygwin, bash doesn't have these in PATH when called from Maven which
+# runs in Windows version of Java.
+export PATH="/bin:/usr/bin:$PATH"
+
+############################################################################
+# Helper functions
+############################################################################
+E_PARAM_ERR=98
+E_ASSERT_FAILED=99
+
+# Usage:
+fail()
+{
+  echo "ERROR: $1"
+  echo
+  exit $E_ASSERT_FAILED
+}
+
+# Usage: assertEq VAL1 VAL2 $LINENO
+assertEq ()
+{
+  lineno=$3
+  if [ -z "$lineno" ]; then
+    echo "lineno not given"
+    exit $E_PARAM_ERR
+  fi
+
+  if [[ "$1" != "$2" ]]; then
+    echo "Assertion failed:  \"$1\" == \"$2\""
+    echo "File \"$0\", line $lineno"    # Give name of file and line number.
+    exit $E_ASSERT_FAILED
+  fi
+}
+
+# Checks the artifact is for the expected architecture
+# Usage: checkArch <path-to-protoc>
+checkArch ()
+{
+  echo
+  echo "Checking file format ..."
+  if [[ "$OS" == windows || "$OS" == linux ]]; then
+    format="$(objdump -f "$1" | grep -o "file format .*$" | grep -o "[^ ]*$")"
+    echo Format=$format
+    if [[ "$OS" == linux ]]; then
+      if [[ "$ARCH" == x86_32 ]]; then
+        assertEq $format "elf32-i386" $LINENO
+      elif [[ "$ARCH" == x86_64 ]]; then
+        assertEq $format "elf64-x86-64" $LINENO
+      else
+        fail "Unsupported arch: $ARCH"
+      fi
+    else
+      # $OS == windows
+      if [[ "$ARCH" == x86_32 ]]; then
+        assertEq $format "pei-i386" $LINENO
+      elif [[ "$ARCH" == x86_64 ]]; then
+        assertEq $format "pei-x86-64" $LINENO
+      else
+        fail "Unsupported arch: $ARCH"
+      fi
+    fi
+  elif [[ "$OS" == osx ]]; then
+    format="$(file -b "$1" | grep -o "[^ ]*$")"
+    echo Format=$format
+    if [[ "$ARCH" == x86_32 ]]; then
+      assertEq $format "i386" $LINENO
+    elif [[ "$ARCH" == x86_64 ]]; then
+      assertEq $format "x86_64" $LINENO
+    else
+      fail "Unsupported arch: $ARCH"
+    fi
+  else
+    fail "Unsupported system: $OS"
+  fi
+  echo
+}
+
+# Checks the dependencies of the artifact. Artifacts should only depend on
+# system libraries.
+# Usage: checkDependencies <path-to-protoc>
+checkDependencies ()
+{
+  if [[ "$OS" == windows ]]; then
+    dump_cmd='objdump -x '"$1"' | fgrep "DLL Name"'
+    white_list="KERNEL32\.dll\|msvcrt\.dll"
+  elif [[ "$OS" == linux ]]; then
+    dump_cmd='ldd '"$1"
+    if [[ "$ARCH" == x86_32 ]]; then
+      white_list="linux-gate\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld-linux\.so\.2"
+    elif [[ "$ARCH" == x86_64 ]]; then
+      white_list="linux-vdso\.so\.1\|libpthread\.so\.0\|libm\.so\.6\|libc\.so\.6\|ld-linux-x86-64\.so\.2"
+    fi
+  elif [[ "$OS" == osx ]]; then
+    dump_cmd='otool -L '"$1"' | fgrep dylib'
+    white_list="libz\.1\.dylib\|libstdc++\.6\.dylib\|libSystem\.B\.dylib"
+  fi
+  if [[ -z "$white_list" || -z "$dump_cmd" ]]; then
+    fail "Unsupported platform $OS-$ARCH."
+  fi
+  echo "Checking for expected dependencies ..."
+  eval $dump_cmd | grep -i "$white_list" || fail "doesn't show any expected dependencies"
+  echo "Checking for unexpected dependencies ..."
+  eval $dump_cmd | grep -i -v "$white_list"
+  ret=$?
+  if [[ $ret == 0 ]]; then
+    fail "found unexpected dependencies (listed above)."
+  elif [[ $ret != 1 ]]; then
+    fail "Error when checking dependencies."
+  fi  # grep returns 1 when "not found", which is what we expect
+  echo "Dependencies look good."
+  echo
+}
+############################################################################
+
+echo "Building protoc, OS=$OS ARCH=$ARCH"
+
+# Nested double quotes are unintuitive, but it works.
+cd "$(dirname "$0")"
+
+WORKING_DIR=$(pwd)
+CONFIGURE_ARGS="--disable-shared"
+
+MAKE_TARGET="protoc"
+if [[ "$OS" == windows ]]; then
+  MAKE_TARGET="${MAKE_TARGET}.exe"
+fi
+
+# Override the default value set in configure.ac that has '-g' which produces
+# huge binary.
+CXXFLAGS="-DNDEBUG"
+LDFLAGS=""
+
+if [[ "$(uname)" == CYGWIN* ]]; then
+  assertEq "$OS" windows $LINENO
+  # Use mingw32 compilers because executables produced by Cygwin compiler
+  # always have dependency on Cygwin DLL.
+  if [[ "$ARCH" == x86_64 ]]; then
+    CONFIGURE_ARGS="$CONFIGURE_ARGS --host=x86_64-w64-mingw32"
+  elif [[ "$ARCH" == x86_32 ]]; then
+    CONFIGURE_ARGS="$CONFIGURE_ARGS --host=i686-pc-mingw32"
+  else
+    fail "Unsupported arch by CYGWIN: $ARCH"
+  fi
+elif [[ "$(uname)" == MINGW32* ]]; then
+  assertEq "$OS" windows $LINENO
+  assertEq "$ARCH" x86_32 $LINENO
+elif [[ "$(uname)" == Linux* ]]; then
+  if [[ "$OS" == linux ]]; then
+    if [[ "$ARCH" == x86_64 ]]; then
+      CXXFLAGS="$CXXFLAGS -m64"
+    elif [[ "$ARCH" == x86_32 ]]; then
+      CXXFLAGS="$CXXFLAGS -m32"
+    else
+      fail "Unsupported arch: $ARCH"
+    fi
+  elif [[ "$OS" == windows ]]; then
+    # Cross-compilation for Windows
+    # TODO(zhangkun83) MinGW 64 always adds dependency on libwinpthread-1.dll,
+    # which is undesirable for repository deployment.
+    CONFIGURE_ARGS="$CONFIGURE_ARGS"
+    if [[ "$ARCH" == x86_64 ]]; then
+      CONFIGURE_ARGS="$CONFIGURE_ARGS --host=x86_64-w64-mingw32"
+    elif [[ "$ARCH" == x86_32 ]]; then
+      CONFIGURE_ARGS="$CONFIGURE_ARGS --host=i686-w64-mingw32"
+    else
+      fail "Unsupported arch: $ARCH"
+    fi
+  else
+    fail "Cannot build $OS on $(uname)"
+  fi
+elif [[ "$(uname)" == Darwin* ]]; then
+  assertEq "$OS" osx $LINENO
+  # Make the binary compatible with OSX 10.7 and later
+  CXXFLAGS="$CXXFLAGS -mmacosx-version-min=10.7"
+  if [[ "$ARCH" == x86_64 ]]; then
+    CXXFLAGS="$CXXFLAGS -m64"
+  elif [[ "$ARCH" == x86_32 ]]; then
+    CXXFLAGS="$CXXFLAGS -m32"
+  else
+    fail "Unsupported arch: $ARCH"
+  fi
+else
+  fail "Unsupported system: $(uname)"
+fi
+
+# Statically link libgcc and libstdc++.
+# -s to produce stripped binary.
+# And they don't work under Mac.
+if [[ "$OS" != osx ]]; then
+  LDFLAGS="$LDFLAGS -static-libgcc -static-libstdc++ -s"
+fi
+
+export CXXFLAGS LDFLAGS
+
+TARGET_FILE=target/protoc.exe
+
+cd "$WORKING_DIR"/.. && ./configure $CONFIGURE_ARGS &&
+  cd src && make clean && make $MAKE_TARGET &&
+  cd "$WORKING_DIR" && mkdir -p target &&
+  (cp ../src/protoc $TARGET_FILE || cp ../src/protoc.exe $TARGET_FILE) ||
+  exit 1
+
+if [[ "$OS" == osx ]]; then
+  # Since Mac linker doesn't accept "-s", we need to run strip
+  strip $TARGET_FILE || exit 1
+fi
+
+checkArch $TARGET_FILE && checkDependencies $TARGET_FILE
diff --git a/protoc-artifacts/pom.xml b/protoc-artifacts/pom.xml
new file mode 100644
index 00000000..fad81b57
--- /dev/null
+++ b/protoc-artifacts/pom.xml
@@ -0,0 +1,134 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+  <modelVersion>4.0.0</modelVersion>
+  <parent>
+    <groupId>com.google</groupId>
+    <artifactId>google</artifactId>
+    <version>1</version>
+  </parent>
+  <groupId>com.google.protobuf</groupId>
+  <artifactId>protoc</artifactId>
+  <version>3.0.0-alpha-3-pre</version>
+  <packaging>pom</packaging>
+  <name>Protobuf Compiler</name>
+  <description>
+    Protobuf Compiler (protoc) is a compiler for .proto files. It generates
+    language-specific code for Protobuf messages and RPC interfaces.
+  </description>
+  <inceptionYear>2008</inceptionYear>
+  <url>https://developers.google.com/protocol-buffers/</url>
+  <licenses>
+    <license>
+      <name>New BSD license</name>
+      <url>http://www.opensource.org/licenses/bsd-license.php</url>
+      <distribution>repo</distribution>
+    </license>
+  </licenses>
+  <scm>
+    <url>https://github.com/google/protobuf</url>
+    <connection>
+      scm:git:https://github.com/google/protobuf.git
+    </connection>
+  </scm>
+  <build>
+    <extensions>
+      <extension>
+        <groupId>kr.motd.maven</groupId>
+        <artifactId>os-maven-plugin</artifactId>
+        <version>1.2.3.Final</version>
+      </extension>
+    </extensions>
+    <plugins>
+      <plugin>
+        <groupId>org.codehaus.mojo</groupId>
+        <artifactId>exec-maven-plugin</artifactId>
+        <version>1.1.1</version>
+        <executions>
+          <execution>
+            <phase>compile</phase>
+            <goals>
+              <goal>exec</goal>
+            </goals>
+          </execution>
+        </executions>
+        <configuration>
+          <executable>bash</executable>
+          <arguments>
+            <argument>build-protoc.sh</argument>
+            <argument>${os.detected.name}</argument>
+            <argument>${os.detected.arch}</argument>
+          </arguments>
+        </configuration>
+      </plugin>
+      <plugin>
+        <groupId>org.codehaus.mojo</groupId>
+        <artifactId>build-helper-maven-plugin</artifactId>
+        <version>1.8</version>
+        <executions>
+          <execution>
+            <id>attach-artifacts</id>
+            <phase>package</phase>
+            <goals>
+              <goal>attach-artifact</goal>
+            </goals>
+            <configuration>
+              <artifacts>
+                <artifact>
+                  <file>${basedir}/target/protoc.exe</file>
+                  <classifier>${os.detected.name}-${os.detected.arch}</classifier>
+                  <type>exe</type>
+                </artifact>
+              </artifacts>
+            </configuration>
+          </execution>
+        </executions>
+      </plugin>
+    </plugins>
+  </build>
+  <profiles>
+    <profile>
+      <id>release</id>
+      <properties>
+        <!-- Specify the staging repository to deploy to. This can be left
+             empty for the first deployment, and Sonatype will create one. For
+             subsequent deployments it should be set to what Sonatype has
+             created, so that all deployments will go to the same repository.
+             -->
+        <staging.repository></staging.repository>
+      </properties>
+      <build>
+        <plugins>
+          <plugin>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-gpg-plugin</artifactId>
+            <version>1.5</version>
+            <executions>
+              <execution>
+                <id>sign-artifacts</id>
+                <phase>verify</phase>
+                <goals>
+                  <goal>sign</goal>
+                </goals>
+              </execution>
+            </executions>
+          </plugin>
+          <plugin>
+            <groupId>org.sonatype.plugins</groupId>
+            <artifactId>nexus-staging-maven-plugin</artifactId>
+            <version>1.6.3</version>
+            <extensions>true</extensions>
+            <configuration>
+               <serverId>sonatype-nexus-staging</serverId>
+               <nexusUrl>https://oss.sonatype.org/</nexusUrl>
+               <skipStagingRepositoryClose>true</skipStagingRepositoryClose>
+               <autoReleaseAfterClose>false</autoReleaseAfterClose>
+               <stagingRepositoryId>${staging.repository}</stagingRepositoryId>
+            </configuration>
+          </plugin>
+        </plugins>
+      </build>
+    </profile>
+  </profiles>
+</project>