Very often when I'm programming in Java, I need to see the dependencies for a particular jar file. If you are working in a project using a build tool like Maven or SBT, it's quite trivial to dump out a dependency tree of all the jars in the project. Outside of the project it's a pain, generally you have to throw together a quick project and then run your tool on it. As a workaround, I threw together a script that does that for you (and cleans up afterwards) Just copy the code into a file named `mvntree`. You should tweak the included repositories to reflect ones that you may need. To run it use:

mvntree groupId artifactId version


For example:

mvntree edu.ucar netcdf4 4.6.9


mvntree

#!/usr/bin/env bash

# Usage: mvntree groupId artifactId version

CUR_DIR=$(pwd)
MY_DIR="/tmp/trashme_one"

mkdir -p "$MY_DIR" && \
  cat > "$MY_DIR/pom.xml" <<EOL
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <groupId>DEPENDENCY</groupId>
   <artifactId>TREE.FOR</artifactId>
   <version>IS</version>
   <packaging>jar</packaging>
   <name>Maven Quick Start Archetype</name>
   <url>http://maven.apache.org</url>
   <dependencies>
      <dependency>
         <groupId>$1</groupId>
         <artifactId>$2</artifactId>
         <version>$3</version>
      </dependency>
<!-- add here other dependencies -->
   </dependencies>
   <repositories>
     <repository>
       <snapshots>
         <enabled>false</enabled>
       </snapshots>
       <id>bintray-hohonuuli-maven</id>
       <name>bintray</name>
       <url>http://dl.bintray.com/hohonuuli/maven</url>
     </repository>
     <repository>
       <snapshots>
         <enabled>false</enabled>
       </snapshots>
       <id>ucar</id>
       <name>ucar</name>
       <url>http://artifacts.unidata.ucar.edu/content/repositories/unidata-releases/</url>
     </repository>
   </repositories>
</project>
EOL

cd "$MY_DIR" && \
mvn -e dependency:tree -U

cd "$CUR_DIR"
rm -rf $MY_DIR 

I'm calculating SHA-512 checksum's for video files that will be registered with a custom video asset manager that I wrote. Checksum's are useful fingerprints for identifying a file. In my case the checksums are useful for doing reverse look ups; e.g. if I have a file, I can look up it's metadata just by using it's checksum.

SHA-512 from Command Line

For my application, I can use the linux/mac util shasum when I register the file. In this case the command looks like:
shasum -a 512 -p /path/to/videofile.mp4
For a 40GB file, this takes about 160 seconds to run. For a 4 GB file, it takes 20 seconds.

SHA-512 in Java

However, I also need to calculate the checksum in a Java/Scala application. To do that I used the following script (written in scala)
#!/usr/bin/env scala

// Output matches the shasum -a 512 -p output

import java.io._
import java.math._
import java.security._


val file = new File(args(0))
val h = hash(file)
println(new BigInteger(1, h).toString(16)) // Dump out hash as hex


def hash(file: File): Array[Byte] = {
  val in = new BufferedInputStream(new FileInputStream(file))
  val digest = MessageDigest.getInstance("SHA-512");
  val buffer = Array.ofDim[Byte](1048576) // 1 MB. I tried 4MB and it was the same
  var sizeRead = -1
  var ok = true
  while(ok) {
    sizeRead = in.read(buffer)
    if (sizeRead == -1) ok = false
    else digest.update(buffer, 0, sizeRead)
  }
  in.close()
  
  digest.digest()
}
This takes slightly longer than the command line, but it's pretty close. A 40GB file takes about 185 seconds, a 4GB file takes about 21 seconds. Of course, that also includes the time to compile the code.
If you upgrade to the latest beta of iTerm (v3-ish), you will find that the iTerm support in TextMate is broken. i.e. Trying to launch an interactive shell from TextMate will fail. I wrote up a little bit of AppleScript to fix this issue.

Step 1

Open Script Editor and create a new doc. Paste the following code into it:

(* 

    Open Terminal Here 
     
    A toolbar script for Mac OS X 10.3/10.4
     
    Written by Brian Schlining
 *)


property debug : false

-- when the toolbar script icon is clicked 
-- 
on run argv
 set this_folder to item 1 of argv
 my process_item(this_folder)
end run



-- this subroutine processes does the actual work 
-- this version can handle this weirdo case: a folder named "te'st"ö te%s`t"

on process_item(this_item)
 
 set thePath to quoted form of POSIX path of this_item
 set theCmd to "cd " & thePath & ";clear;"
 
 tell application "iTerm"
  activate
  -- just open a terminal and cd to thePath
  --set newWindow to (create window with default profile)
  
  set aWindow to current window
  
  if aWindow is equal to missing value then
   set aWindow to (create window with default profile)
  else
   tell aWindow
    set t to (create tab with default profile)
   end tell
  end if
  
  
  tell current session of aWindow
   write text "cd " & thePath & ";clear;"
  end tell
  
 end tell
 
 
end process_item
Save the script somewhere as iTermLaunch.scpt.

Step 2

In TextMate, select "Bundles -> Edit Bundles" from the menu. Then select the "iTerm" bundle. Select Menu Actions in the iTerm bundle. Select "Interactive Shell" and edit the script like so (replace "/Path/to" with the real path to your iTermLaunch.scpt

#!/bin/bash
[[ -f "${TM_SUPPORT_PATH}/lib/bash_init.sh" ]] && . "${TM_SUPPORT_PATH}/lib/bash_init.sh"

# Start a new shell if needed, otherwise show the one with the right name.
#export SHELL_NAME=${SHELL_NAME:="TextMate Shell"}
#"$TM_BUNDLE_SUPPORT/new.sh"

osascript "/Path/to/iTermLaunch.scpt" ${TM_PROJECT_DIRECTORY}

Remember to save the bundle ([CMD]+S)

Step 3

There is no step 3. That's it. Just run that bundle item to open an iTerm shell in your current project.
I just got a notice that iTerm2 has new major upcoming release (v. 3). I upgraded but found that the AppleScript dialect that iTerm used has changed. This change broke my handy little script that launches iTerm from a Finder window. So I took a few minutes to update it. The new code for iTerm2 v3 is below. Just copy and paste it into a new ScriptEditor window and save as iTermHere.scpt. You can then export it as an Application from ScriptEditor and drag it to your Finder toolbar. Here's the code:

(* 

    Open Terminal Here 
     
    A toolbar script for Mac OS X 10.3/10.4
     
    Written by Brian Schlining
 *)


property debug : false

-- when the toolbar script icon is clicked 
-- 
on run
 tell application "Finder"
  try
   set this_folder to (the target of the front window) as alias
  on error
   set this_folder to startup disk
  end try
  
  my process_item(this_folder)
  
 end tell
end run


-- This handler processes folders dropped onto the toolbar script icon 
-- 
on open these_items
 repeat with i from 1 to the count of these_items
  set this_item to item i of these_items
  my process_item(this_item)
 end repeat
end open


-- this subroutine processes does the actual work 
-- this version can handle this weirdo case: a folder named "te'st"ö te%s`t"

on process_item(this_item)
 
 set thePath to quoted form of POSIX path of this_item
 set theCmd to "cd " & thePath & ";clear;"
 
 tell application "iTerm"
  activate
  -- just open a terminal and cd to thePath
  --set newWindow to (create window with default profile)
  
  set aWindow to current window
  
  if aWindow is equal to missing value then
   set aWindow to (create window with default profile)
  else
   tell aWindow
    set t to (create tab with default profile)
   end tell
  end if
  
  
  tell current session of aWindow
   write text "cd " & thePath & ";clear;"
  end tell
  
 end tell
 
 
end process_item

My goto editor these days is TextMate. (The recent 2.0 releases have been really great). Like many developers, I use git for source control. Most of the time I use git on the command line, but once in a while I'll open up a project in the very awesome, and very free SourceTree VCS Gui from Atlassian. To simplify opening a project in SourceTree when working from TextMate I added a command via TextMate's extremely simple and flexible Bundles. (Which, due to the simplicity and flexibility of bundle's in TextMate, just tromps on anything I've seen in any other text editor, sorry Sublime Text.).

First, you will need to install SourceTree. Once installed, launch it and select SourceTree > Install Command Line Tools to install the stree shell command.

To add the command to TextMate, do the following:

  1. In TextMate, go to Bundles > Edit Bundles
  2. Select the Bundle that you want to add your command to. I created one with my own name, but you can use existing ones.
  3. Hit [command]+N to open a dialog to Create a New Item
  4. In the Create New Item dialog choose Command
  5. In the sidebar, name it Open With SourceTree. Add your favorite key equivalent (I use [command]+. )
  6. Set Save to Nothing
  7. Set Input to Nothing
  8. Set Output to Discard
  9. In the script shell at the bottom enter the following:
    
    #!/usr/bin/env bash
    if [ -d $TM_DIRECTORY ] ; then
      cd $TM_DIRECTORY 
    elif [ -d $TM_PROJECT_DIRECTORY ] ; then
      cd $TM_PROJECT_DIRECTORY
    fi
      
    GIT_ROOT=$(git rev-parse --show-toplevel)
    if [ -d $GIT_ROOT ] ; then
      stree $GIT_ROOT
    fi
      
And that's it. Enjoy.

At this moment ScalaDays is going on in nearby San Francisco. I came across a Toward a Safer Scala talk that emphasized writing safer Scala using strict static code analysis. After watching the talk I put together a giter8 template that implements their recommendations. If you have giter8 installed you can try out the basicscala template using:

g8 hohonuuli/basicscala.g8
Here's some things that this template provides:
  • Formats source files on compile using Scalariform
  • Adds strict scala compiler flags in build.sbt
  • Build fails on compiler warnings
  • Adds Wartremover to Scala's linter. You can run lint and wartremover in sbt using 'lint:compile'.
  • Includes junit and scalatest
  • Includes scala-style for checking code sytle. You will need to run 'sbt scalastyleGenerateConfig' to generate a scalastyle-config file. After which, you can run 'sbt scalastyle' to check your projects.
I'm currently working on a project that has to watermark a few hundred thousand images and add some EXIF metadata to the images. For various reasons, JVM-based languages work best for this problem at work, so I've spent some time learning Apache's commons-imaging libraries as they support reading and writing EXIF metadata in Java. Using exiftool as 'truth', I worked through writing the metadata needed using first exiftool then commons-imaging and comparing the results. I ran into a few issues related to TIFF metadata directories that I needed to debug. In, order to do this I created a Scala script that dumps out EXIF info along with the tag id and directory id. In order to get this example to work you will need the following:
  1. Install SBT. On Macs, you can use Homebrew to install it.
  2. Create a shell script named scalas with the following contents: scalas
    
    #!/usr/bin/env bash
    
    # This script calls SBT and executes a scala script that requires external dependencies
    # The build details are added as a comment at the top of the script
    # You can set SBT_HOME or hard code the path to the directory where SBT is installed.
    # If you installed SBT using homebrew then add this to your .bashrc file:
    #
    # export SBT_HOME='/usr/local/opt/sbt/libexec'
    
    java -Dsbt.main.class=sbt.ScriptMain -Dsbt.boot.directory=$HOME/.sbt/boot -jar $SBT_HOME/sbt-launch.jar "$@"
       
Using scalas we can run scripts with SBT settings passed in. This allows us to link to 3rd party jars from our scripts without having to hard code paths. Now create a file named readexif and add the following:
 
#!/usr/bin/env scalas

/*
 Dump out exif metadata

 Usage: readexif filename
*/

/***
scalaVersion := "2.11.5"

libraryDependencies ++= Seq(
  "org.apache.commons" % "commons-imaging" % "1.0-SNAPSHOT")

resolvers in ThisBuild ++= Seq(Resolver.mavenLocal,
    "mbari-maven-repository" at "https://mbari-maven-repository.googlecode.com/svn/repository",
    "Apache Development Snapshot Repository" at "https://repository.apache.org/content/repositories/snapshots/")

*/

import org.apache.commons.imaging.common.GenericImageMetadata.GenericImageMetadataItem
import org.apache.commons.imaging.common.ImageMetadata.ImageMetadataItem
import org.apache.commons.imaging.formats.jpeg.JpegImageMetadata
import org.apache.commons.imaging.formats.tiff.TiffImageMetadata.TiffMetadataItem
import org.apache.commons.imaging.Imaging
import java.io.File
import scala.collection.JavaConverters._

val file = new File(args(0))
val metadata = Imaging.getMetadata(file)

if (metadata == null) {
  println("\tNo image metadata was found")
  System.exit(0)
}

println(s"  -- ALL EXIF info (using ${metadata.getClass.getName})")
println(s"  (tag    -- directory )")
metadata match {
  case j: JpegImageMetadata => dump(j)
}

def dump(j: JpegImageMetadata): Unit = {
  j.getItems.asScala.foreach {i => 
    i match {
      case g: GenericImageMetadataItem => printItem(g)
      case _ => print("      $i")
    }
  }
}

def printItem(i: GenericImageMetadataItem): Unit = {
  val s = s"${i.getKeyword} : ${i.getText}"
  val tagString = i match {
    case tmi: TiffMetadataItem => {
      val tmi = i.asInstanceOf[TiffMetadataItem]
      val tiffField = tmi.getTiffField
      val tag = tiffField.getTag
      val directoryType = tiffField.getDirectoryType
      f"  (0x${tag}%04x -- 0x${directoryType}%08x)  $s"
    }
    case _ => s
  }
  println(tagString)
}

Here's example output from readexif somefile.jpg:
  -- ALL EXIF info (using org.apache.commons.imaging.formats.jpeg.JpegImageMetadata)
  (tag    -- directory )
  (0x8298 -- 0x00000000)  Copyright : 'Copyright 2003 Monterey Bay Aquarium Research Institute'
  (0x8769 -- 0x00000000)  ExifOffset : 108
  (0x8825 -- 0x00000000)  GPSInfo : 192
  (0x882a -- 0xfffffffe)  TimeZoneOffset : 0
  (0x9003 -- 0xfffffffe)  DateTimeOriginal : '2014:01:28 10:11:12'
  (0x9004 -- 0xfffffffe)  DateTimeDigitized : '2015:01:01 00:00:00'
  (0x0001 -- 0xfffffffd)  GPSLatitudeRef : 'N'
  (0x0002 -- 0xfffffffd)  GPSLatitude : 40, 43, 1/2147483647 (0)
  (0x0003 -- 0xfffffffd)  GPSLongitudeRef : 'W'
  (0x0004 -- 0xfffffffd)  GPSLongitude : 74, 0, 0
  (0x0005 -- 0xfffffffd)  GPSAltitudeRef : 1
  (0x0006 -- 0xfffffffd)  GPSAltitude : 4203/10 (420.3)
  (0x001b -- 0xfffffffd)  GPSProcessingMethod : 'MANUAL'
Compared to the output of exiftool somefile.jpg:
ExifTool Version Number         : 9.76
File Name                       : WriteExif01$-01_45_25_18-external.jpg
Directory                       : target
File Size                       : 56 kB
File Modification Date/Time     : 2015:02:04 10:53:09-08:00
File Access Date/Time           : 2015:02:04 13:56:01-08:00
File Inode Change Date/Time     : 2015:02:04 10:53:09-08:00
File Permissions                : rw-r--r--
File Type                       : JPEG
MIME Type                       : image/jpeg
JFIF Version                    : 1.02
Resolution Unit                 : None
X Resolution                    : 1
Y Resolution                    : 1
Exif Byte Order                 : Little-endian (Intel, II)
Copyright                       : Copyright 2003 Monterey Bay Aquarium Research Institute
Time Zone Offset                : 0
Date/Time Original              : 2014:01:28 10:11:12
Create Date                     : 2015:01:01 00:00:00
GPS Version ID                  : 2.3.0.0
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude Ref                : Below Sea Level
GPS Processing Method           : MANUAL
Image Width                     : 640
Image Height                    : 486
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
GPS Altitude                    : 420.3 m Below Sea Level
GPS Latitude                    : 40 deg 43' 0.00" N
GPS Longitude                   : 74 deg 0' 0.00" W
GPS Position                    : 40 deg 43' 0.00" N, 74 deg 0' 0.00" W
Image Size                      : 640x486
I needed to create a small test image with known pixel values. The easiest way for me was to fire up Matlab and use it's built in commands to generate a small PNG image. I'm just documenting the steps here for the next time I need to do such a thing.

The Code

Here's the code:

%% Creating a 20x20 pixel image

% Here's a linear ramp of 400 samples
r = 0:399;

% reshape the ramp to be a row-oriented 20x20 matrix
% i.e. r(1, 1) = 0, r(1, 2) = 1 ...
r = reshape(r, 20, 20)';

% Create other color arrays
g = r / 2;
b = flipud(fliplr(r));

% Stuff them all into one 20x20x3 matrix
i = zeros(20, 20, 3);
i(:, :, 1) = r;
i(:, :, 2) = g;
i(:, :, 3) = b;

% Normalize to byte values
ui = uint8(i / max(max(max(i))) * 255); 

% write it out
imwrite(ui, 'smalltestimage.png', 'PNG');

The Image

And Voila ...

The Pixel Values

And for the record here's the actual pixel values in ui

>> ui

ui(:,:,1) =

    0    1    1    2    3    3    4    4    5    6    6    7    8    8    9   10   10   11   12   12
   13   13   14   15   15   16   17   17   18   19   19   20   20   21   22   22   23   24   24   25
   26   26   27   27   28   29   29   30   31   31   32   33   33   34   35   35   36   36   37   38
   38   39   40   40   41   42   42   43   43   44   45   45   46   47   47   48   49   49   50   50
   51   52   52   53   54   54   55   56   56   57   58   58   59   59   60   61   61   62   63   63
   64   65   65   66   66   67   68   68   69   70   70   71   72   72   73   73   74   75   75   76
   77   77   78   79   79   80   81   81   82   82   83   84   84   85   86   86   87   88   88   89
   89   90   91   91   92   93   93   94   95   95   96   97   97   98   98   99  100  100  101  102
  102  103  104  104  105  105  106  107  107  108  109  109  110  111  111  112  112  113  114  114
  115  116  116  117  118  118  119  120  120  121  121  122  123  123  124  125  125  126  127  127
  128  128  129  130  130  131  132  132  133  134  134  135  135  136  137  137  138  139  139  140
  141  141  142  143  143  144  144  145  146  146  147  148  148  149  150  150  151  151  152  153
  153  154  155  155  156  157  157  158  158  159  160  160  161  162  162  163  164  164  165  166
  166  167  167  168  169  169  170  171  171  172  173  173  174  174  175  176  176  177  178  178
  179  180  180  181  182  182  183  183  184  185  185  186  187  187  188  189  189  190  190  191
  192  192  193  194  194  195  196  196  197  197  198  199  199  200  201  201  202  203  203  204
  205  205  206  206  207  208  208  209  210  210  211  212  212  213  213  214  215  215  216  217
  217  218  219  219  220  220  221  222  222  223  224  224  225  226  226  227  228  228  229  229
  230  231  231  232  233  233  234  235  235  236  236  237  238  238  239  240  240  241  242  242
  243  243  244  245  245  246  247  247  248  249  249  250  251  251  252  252  253  254  254  255


ui(:,:,2) =

    0    0    1    1    1    2    2    2    3    3    3    4    4    4    4    5    5    5    6    6
    6    7    7    7    8    8    8    9    9    9   10   10   10   11   11   11   12   12   12   12
   13   13   13   14   14   14   15   15   15   16   16   16   17   17   17   18   18   18   19   19
   19   19   20   20   20   21   21   21   22   22   22   23   23   23   24   24   24   25   25   25
   26   26   26   27   27   27   27   28   28   28   29   29   29   30   30   30   31   31   31   32
   32   32   33   33   33   34   34   34   35   35   35   35   36   36   36   37   37   37   38   38
   38   39   39   39   40   40   40   41   41   41   42   42   42   43   43   43   43   44   44   44
   45   45   45   46   46   46   47   47   47   48   48   48   49   49   49   50   50   50   50   51
   51   51   52   52   52   53   53   53   54   54   54   55   55   55   56   56   56   57   57   57
   58   58   58   58   59   59   59   60   60   60   61   61   61   62   62   62   63   63   63   64
   64   64   65   65   65   66   66   66   66   67   67   67   68   68   68   69   69   69   70   70
   70   71   71   71   72   72   72   73   73   73   73   74   74   74   75   75   75   76   76   76
   77   77   77   78   78   78   79   79   79   80   80   80   81   81   81   81   82   82   82   83
   83   83   84   84   84   85   85   85   86   86   86   87   87   87   88   88   88   89   89   89
   89   90   90   90   91   91   91   92   92   92   93   93   93   94   94   94   95   95   95   96
   96   96   97   97   97   97   98   98   98   99   99   99  100  100  100  101  101  101  102  102
  102  103  103  103  104  104  104  104  105  105  105  106  106  106  107  107  107  108  108  108
  109  109  109  110  110  110  111  111  111  112  112  112  112  113  113  113  114  114  114  115
  115  115  116  116  116  117  117  117  118  118  118  119  119  119  120  120  120  120  121  121
  121  122  122  122  123  123  123  124  124  124  125  125  125  126  126  126  127  127  127  128


ui(:,:,3) =

  255  254  254  253  252  252  251  251  250  249  249  248  247  247  246  245  245  244  243  243
  242  242  241  240  240  239  238  238  237  236  236  235  235  234  233  233  232  231  231  230
  229  229  228  228  227  226  226  225  224  224  223  222  222  221  220  220  219  219  218  217
  217  216  215  215  214  213  213  212  212  211  210  210  209  208  208  207  206  206  205  205
  204  203  203  202  201  201  200  199  199  198  197  197  196  196  195  194  194  193  192  192
  191  190  190  189  189  188  187  187  186  185  185  184  183  183  182  182  181  180  180  179
  178  178  177  176  176  175  174  174  173  173  172  171  171  170  169  169  168  167  167  166
  166  165  164  164  163  162  162  161  160  160  159  158  158  157  157  156  155  155  154  153
  153  152  151  151  150  150  149  148  148  147  146  146  145  144  144  143  143  142  141  141
  140  139  139  138  137  137  136  135  135  134  134  133  132  132  131  130  130  129  128  128
  127  127  126  125  125  124  123  123  122  121  121  120  120  119  118  118  117  116  116  115
  114  114  113  112  112  111  111  110  109  109  108  107  107  106  105  105  104  104  103  102
  102  101  100  100   99   98   98   97   97   96   95   95   94   93   93   92   91   91   90   89
   89   88   88   87   86   86   85   84   84   83   82   82   81   81   80   79   79   78   77   77
   76   75   75   74   73   73   72   72   71   70   70   69   68   68   67   66   66   65   65   64
   63   63   62   61   61   60   59   59   58   58   57   56   56   55   54   54   53   52   52   51
   50   50   49   49   48   47   47   46   45   45   44   43   43   42   42   41   40   40   39   38
   38   37   36   36   35   35   34   33   33   32   31   31   30   29   29   28   27   27   26   26
   25   24   24   23   22   22   21   20   20   19   19   18   17   17   16   15   15   14   13   13
   12   12   11   10   10    9    8    8    7    6    6    5    4    4    3    3    2    1    1    0

I attended Big Data Tech Con just this past week. Hadoop is still king of the Big Data World, but Spark is quickly becoming the tool of choice. I attended several very good classes on Spark at the conference. Some of these classes used Scala (via sbt) for the Spark exercises. One of the classes focused on using Spark for machine learning with the exercises in iPython notebooks. This was pretty cool for learning purposes. However, I much prefer working in Scala to Python. Fortunatly, there's the IScala project which allows you to run Scala code in iPython notebooks. I put together an example project on github for those who are interested in trying Spark/Scala notebooks at https://github.com/hohonuuli/sparknotebook. Here's a screenshot of the same code in python and Scala running side by side:
Matlab is an excellent tool for ad-hoc analysis and prototyping. If you store your data in a relational database, Mathworks provides a database toolbox for fetching your data. However, you do not need to purchase this to interact with your database. As Matlab has excellent Java integration, you can use JDBC directly to talk to pretty much any database you want. In order to do this do the following:

Get a JDBC Driver

First, you will need the JDBC driver required to talk to your database. If you do not know how to find the driver, follow this link for an example on how to locate your driver. Once, you have your JDBC driver, you will need to add it to Matlab's static classpath. If you are running an older version of Matlab, say earlier than R2013b, you probably need to add the full path to the driver in the classpath.txt file. You can edit this by typing open classpath.txt in Matlab. If the file says "Do Not Edit This File" at the top then you will need to add the full path to the driver in ~/.matlab/[matlab version]/javaclasspath.txt. (e.g. ~/.matlab/R2014b/javaclasspath.txt). You may need to create this file if it doesn't exist.

Figure out your Database URL

Next, you'll need to figure out the URL to your database. Refer to your drivers documentation. Here's a few example URLS:
Server Driver Jar JDBC URL
SQL Server sqljdbc4.jar jdbc:sqlserver://servername.mydomain.org;database=MyDatabase
SQL Server jtds-1.3.1.jar jdbc:jtds:sqlserver://servername.mydomain.org:1433/MyDatabase
PostgreSQL postgresql-9.3-1102.jdbc41.jar jdbc:postgresql://servername.mydomain.org/MyDatabase

Matlab Code

Now all you need is some Matlab code to fetch your data.

database.m

First, here's a function to setup a database connection:

function c = database(url, user, password, driverName)
% DATABASE - connect to a SQL database
%
% Usage:
%   c = database(url, user, password, driverName)
% 
% Inputs:
%   url = The database URL. e.g. jdbc:jtds:sqlserver://solstice.shore.mbari.org:1433/EXPD
%   user = The user name to connect to the database
%   password = THe password for user
%   driverName = the name of the JDBC driver to use for the connection. 
%       e.g. 'net.sourceforge.jtds.jdbc.Driver' (optional)
%
% Outputs
%   c = A java.sql.Connection object to your database. Remember to call
%   close, either as 'close(c)' or c.close() when you are done with the 
%   connection.
%
% NOTES: In order to connect to a database, you will need the appropriate
% JDBC driver on the classpath. Most drivers do not work on Matlab's dynamic
% classpath. Instead add them to ~/.matlab/R2014b/javaclasspath.txt

% Brian Schlining
% 2014-10-08

if nargin == 4
    d = eval(driverName);
end
c = java.sql.DriverManager.getConnection(url, user, password);

jdbcquery.m

This function runs a query and returns the results as a Matlab structure. Each column of the ResultSet is represented as a separate field in the structure:

function s = jdbcquery(conn, sqlstr)
% Connect to database, execute SQL string, and pass back structure array.
%
% Usage:
%   s = jdbcquery(conn, sqlstr)
% 
% Input   
%   conn: A java.sql.Connection object. See Also database
%   sqlstr: SQL string, i.e. 'SELECT * FROM Observation WHERE ConceptName LIKE ''Pandalus%'''
%
% Output  
%   s: structure array, each field matches the column names in the query.
%
% Note: 
%   The Java class path to the sqljdbc driver must be specified before call.
%   For best results put it in your ~/.matlab/R2014b/javaclasspath.txt file.  
%   This path will vary depending on your Matlab version.
%   
% See also database

% Brian Schlining
% 2014-10-08 - Modified Reiko's version used for EXPD queries
% 2014-11-01 - Using UTC calendar to read dates to avoid timezone issues.

%% Java Imports
import java.sql.ResultSet;

cal = java.util.Calendar.getInstance();
cal.setTimeZone(java.util.TimeZone.getTimeZone('UTC'));

%% Fetch Loop
try    
    % query database
    q = conn.prepareStatement(sqlstr, ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
    rs = q.executeQuery();

    metaData = rs.getMetaData();
    columnCount = metaData.getColumnCount();
    n = resultSetSize(rs);
    s = initStruct(metaData, n);
    
    row = 0; 
    % assign variables from resultset
    while rs.next()
        row = row + 1;
        for i = 1:columnCount
            coltype = char(metaData.getColumnTypeName(i));      % assess the column type
            colName = char(metaData.getColumnName(i));
            switch coltype
                case {'bigint', 'int', 'int4', 'decimal', 'double', 'float',  ...
                        'float8', 'float16', 'real', 'serial'}
                    s.(colName)(row) = toDouble(rs.getObject(i));
                case {'date'}
                    s.(colName)(row) = toDate(rs.getDate(i, cal));
                case {'datetime'}
                    s.(colName)(row) = toDate(rs.getTimestamp(i, cal));
                case {'geometry'}
                    s.(colName){row} = rs.getObject(i);
                otherwise % value is a string, so just pass it into the output array
                    s.(colName){row} = toString(rs.getString(i));
            end
        end
    end 

    % close all database connections
    try 
        q.close();     
    catch
        % DO NOTHING ON EXCEPTION
    end
    try 
        rs.close();     
    catch
        % DO NOTHING ON EXCEPTION
    end

catch me
    ex = MException('JDBC:jdbcquery', ['An error occurred while executing the SQL: \n' sqlstr ]);
    ex = ex.addCause(me);
    ex.throw;
end
end


%%
function s = resultSetSize(resultSet)
    ok = resultSet.last();
    if ~ok
        s = 0;
    else
        s = resultSet.getRow();
        resultSet.beforeFirst();
    end
end

%%
function s = initStruct(metaData, rows)
    for i = 1:metaData.getColumnCount()
       colType = char(metaData.getColumnTypeName(i));
       colName = char(metaData.getColumnName(i));
       switch char(colType)
           case {'bigint', 'int', 'int4', 'decimal', 'double', 'float',  ...
                    'float8', 'float16', 'real', 'serial' 'date', 'datetime'}
               s.(colName) = ones(rows, 1);
           otherwise
               s.(colName) = cell(rows, 1);
       end
    end
end

%%
function v = toDouble(n)
    if isempty(n)
        v = NaN;
    else
        v = double(n);
    end
end

%%
function v = toDate(n)
    if isempty(n)
        v = NaN;
    else
        %datenum('01 Jan 1970 00:00:00') = 719529
        v = n.getTime() / 1000 / 60 / 60 / 24 + 719529;
    end
end

%%
function v = toString(n)
    if isempty(n)
        v = '';
    else
        v = char(n);
    end
end

Putting it all together

Finally, you can use it in Matlab like this:

c = database('jdbc:jtds:sqlserver://severname.mydomain.org:1433/mydatabase', 'someusername', 'supersecretpassword');
sql = 'SELECT * FROM SomeTable';
v = jdbcquery(c, sql);
c.close();
If you query a database often you can dump that all into a function for that particular database. For example:

mydbquery.m


function v = mydbquery(sql)

c = database('jdbc:jtds:sqlserver://servername.mydomain.org:1433/mydatabase', 'someusername', 'supersecretpassword');
v = jdbcquery(c, sql);
c.close();
And then you would just use it as a one liner:

>> v = mydbquery('SELECT * FROM Observation WHERE ConceptName LIKE ''Pandalus%''')

v = 

                   id: [1989x1 double]
      VideoFrameID_FK: [1989x1 double]
       ObservationDTG: [1989x1 double]
             Observer: {1989x1 cell}
          ConceptName: {1989x1 cell}
                Notes: {1989x1 cell}
    LAST_UPDATED_TIME: [1989x1 double]
                    X: [1989x1 double]
                    Y: [1989x1 double]
              rowguid: {1989x1 cell}

Here's a UML diagram of the important classes in the new java.time package in Java 8. It's replacing the very annoying to work with Date class. The new API is derived from work from Joda time thence JSR-310. It has some nice features, but there sure are a lot of parts to it. To help keep track of what's where, here's a UML diagram
imglib2 is a library that I keep coming back to for image manipulation. It's very powerful but way, Way, WAY over abstracted. The documentation for it is just terrible; figuring out how to do simple things is much, much harder than it really needs to be. So, I'm adding a few notes for how to do the simple things. For the examples below I used Scala and SBT. If you use Java the code is very similar and you can use Maven with the same dependencies. You will need the following dependencies in your project:
    "net.imglib2" % "imglib2" % "2.0.0-beta-26",
    "net.imglib2" % "imglib2-ij" % "2.0.0-beta-26",
    "io.scif" % "scifio" % "0.16.2-SNAPSHOT"

Reading images

The correct way(s) to read images with imglib2 seems to change every time I revisit the library. I think the correct way is to use SCIFIO to read images as ImgPlus objects. You can do this like so:
import io.scif.img.IO
import scala.collection.JavaConverters._

...

// Open the image as ImgPlus
val imgs = IO.openImgs(myImagePath).asScala  // openImgs returns a java list, convert to scala
val imgPlus = imgs.head                      // Only 1 item in list so take it.

Getting the Image size

First step is to get the number of dimensions in the image.
val img = imgPlus.getImg                     // imgPlus wraps An Img object which contains the data
val dims = img.numDimensions                 // Returns dimensions
If myImagePath pointed to a typically PNG or JPEG then img will be an ArrayImg object with 3 dimensions in the order of HEIGHT, WIDTH, # COLOR CHANNELS (i.e. 3 for RGB). So to get the size of the image, I iterate over each dimension asking for it's size. In Scala, this is done like so:
// My example RGB image was 2448 pixels wide by 3264 pixels tall. So size
// here would be in the order of (height, width, colorChannels) or (3264, 2448, 3)
val size = for (i <- 0 until dims) yield img.dimension(i)

Navigating over the pixels

imglib2 provides cursors for iterating over the pixels. While this is very fast, the order of iteration is unknown to the programmer. I often want to access a specific pixel in a specific order. The way to do this is to grab a reference to some random access type pointing to the image data and use that to navigate to pixels.
val cursor = img.randomAcess()

// Here we'll walk over all the pixels in the first color channel (which 
// I assume is Red, as in RGB, but I haven't verified that yet)

for {
  w <- 0 until img.dimension(1).toInt // dimensions return longs not ints
  h <- 0 until img.dimension(0).toInt
} {
  val idx = Array(h, w, c)
  cursor.setPosition(idx) // Navigate to desired pixel
  val v = cursor.get      // read pixel value

}
I've recently started playing around with Spark and have decided to give SBT another try. Now normally, I use Maven for building most of my Java and Scala projects. Because of this, I'd like any projects that I build using SBT to work seamlessly with my Maven projects and also, my sbt project should have the same style layout as Maven projects (Just to avoid a sort of build cognitive dissonance). Here's my wish list for my script that generates a default SBT project: It should ...
  • create the standard source and test directory trees
  • create the project/build.properties file
  • create the build.sbt file. The build.sbt file should ...
    • use the local maven repository (~/.m2/repository) as a resolver
    • publish to the local maven repository so my other maven projects can use the project.
    • set the scala version for any sub-projects that are created.
    • include the libraries for JUnit and ScalaTest
  • create a README.md file
  • create a .gitignore file.
  • create a placeholder class and test.
This is what the final project layout will look like:
.
├── .gitignore
├── README.md
├── build.sbt
├── project
│   └── build.properties
└── src
    ├── main
    │   ├── resources
    │   └── scala
    │       └── Main.scala
    └── test
        ├── resources
        └── scala
            └── ExampleSpec.scala

8 directories, 6 files
As SBT doesn't come with scripts that generate projects, I rolled my own. Here's a shell script (Sorry Windows Users) that does the above:

#!/usr/bin/env bash

# Generates a standard layout SBT project

PRJ_DIR=$1

# Build project Structure
mkdir $PRJ_DIR
cd  $PRJ_DIR
mkdir -p src/main/scala
mkdir -p src/main/resources
mkdir -p src/test/scala
mkdir -p src/test/resources

# Write project/build.properties
mkdir -p project
echo "sbt.version=0.13.5" > project/build.properties

# Write build.sbt
cat > build.sbt << EOL

organization := "org.mbari"

name := "$PRJ_DIR"

version := "1.0-SNAPSHOT"

scalaVersion in ThisBuild := "2.10.4"

libraryDependencies ++= Seq(
    "junit" % "junit" % "4.11" % "test",
    "org.scalatest" %% "scalatest" % "2.2.1" % "test"
)

resolvers += Resolver.mavenLocal

publishTo := Some(Resolver.file("file",  new File(Path.userHome.absolutePath+"/.m2/repository")))
EOL

# Write a placeholder class
cat > src/main/scala/Main.scala << EOL
object Main extends App {
  println("Hello World")
}
EOL

# Write a placeholder spec/test
cat > src/test/scala/ExampleSpec.scala << EOL
import collection.mutable.Stack
import org.scalatest._

class ExampleSpec extends FlatSpec with Matchers {

  "A Stack" should "pop values in last-in-first-out order" in {
    val stack = new Stack[Int]
    stack.push(1)
    stack.push(2)
    stack.pop() should be (2)
    stack.pop() should be (1)
  }

  it should "throw NoSuchElementException if an empty stack is popped" in {
    val emptyStack = new Stack[Int]
    a [NoSuchElementException] should be thrownBy {
      emptyStack.pop()
    } 
  }
}
EOL

# Write a README.md
cat > README.md << EOL
# $PRJ_DIR
EOL

# Write a .gitignore file
cat > .gitignore << EOL
*.class
.DS_Store
*.orig

# Package Files #
*.jar
*.war
*.ear

# Build Directory #
target/

# Ignore IntelliJ files
.idea/
*.iml

# eclipse conf file
.settings
.classpath
.project
.manager
.scala_dependencies
EOL

I am playing with Java 8's lambdas. In the past, Java has required that parameters 'closed over' by inner classes need to be final. With Java 8's lambdas, closed over parameters no longer need to be explicitly declared final, but they still need to be effectively final. Here's an example:



import java.util.function.*;

class LambdaTest {
  public static void main(String[] args) {
    Consumer fn = (Void) -> { System.out.println("One"); };
    Runnable r = () -> { fn.accept(null); };
    r.run();
  }
}


fn is closed over by the Runnable that we created and runs as expected producing the output of "One". However, if we try to change the value of fn the compiler will complain:



import java.util.function.*;

class LambdaTest2 {
  public static void main(String[] args) {
    Consumer fn = (Void) -> { System.out.println("One"); };
    Runnable r = () -> { fn.accept(null); };
    r.run();
    fn = (Void) -> { System.out.println("Two"); }; // compiler barfs
    r.run();
  }
}


Since we've changed the value of fn the compiler will respond with: error: local variables referenced from a lambda expression must be final or effectively final. The workaround for this is to indirectly call fn via a method:



import java.util.function.*;

class LambdaTest3 {
  
  private static Consumer fn;
  
  public static void main(String[] args) {
    fn = (Void) -> { System.out.println("One");};
    // Do not call fn directly, use a method call instead.
    Runnable r = () -> {callFn();};
    r.run();
    fn = (Void) -> {System.out.println("Two");};
    r.run();
  }
  
  private static void callFn() {
    fn.accept(null);
  }
 
}


Now we get the output that we were expecting; the above programs will print "One" then "Two".
I'm the author of a project for managing video annotations. The bulk of our videos are stored on video tapes. However, we are slooooowly moving over to video file storage. Our annotation system, which is written in Java, does work with video files, but via the very, very antiquated and deprecated QuickTime for Java libraries. I'm currently exploring alternatives to beef up video files support. Current candidates include JavaFX, AVFoundation (via JNI) and VLC (via its rc interface). Each of these options has strengths and weaknesses.

VLC Strengths

  • Works on the platforms I care about (Mac OS X, Linux, and Windows)
  • Has a very capable remote interface that can be accessed via TCP. Seems pretty straightforward to implement
  • Supports a wide variety of codecs and media containers
  • Supports frame capture (snapshots) from videos

VLC Weaknesses

  • Not Java, so there will be some configuration and install hassles for users
  • Java will need to launch it via Runtime.getRuntime().exec("..."); and manage the external process.
  • The command for launching VLC is different for different operating systems
  • Near as I can tell, it doesn't support timecode tracks. This will be a problem for my organization.
  • Lousy time resolution when indexing media remotely; it only indexes to the nearest second.

Example code

This is just a quick example. I'm working on a Mac so for simplicity I launched VLC from a terminal window using:

/Applications/VLC.app/Contents/MacOS/VLC --extraintf rc --rc-host=localhost:8080

The above command launches the default user interface (called macosx on Macs) as well as the rc network interface. In the future, I'll expect to use that command to launch VLC directly from Java.


Here's a bit of Java code for interacting with VLC:


import java.net.InetAddress;
import java.net.Socket;
import java.io.*;

// VLC RC Command references/examples
//   - https://github.com/mguinada/vlc-client/blob/master/lib/vlc-client/client/media_controls.rb
//   - http://getluky.net/2006/04/19/vlcs-awesome-rc-interface/
class VLCTest {
  
  private static BufferedReader in;
  private static OutputStream out;
  
  public static void main(String[] args) throws Exception {
    InetAddress localhost = InetAddress.getByName("localhost");
    Socket socket = new Socket(localhost, 8080);
    out = socket.getOutputStream();
    in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
    // Just read responses from VLC in a seperate thread.
    Thread readerThread = new Thread(new Runnable() {
      public void run() {
        boolean ok = true;
       while(ok) {
         try {
           System.out.println(in.readLine());           
         }
         catch (Exception e) {
           System.out.println("Whoops: " + e.getMessage());
           ok = false;
         }
       } 
      }
    });
    readerThread.setDaemon(true);
    readerThread.start();
    out.write("clear\n".getBytes());

    // !!!! CHANGE THIS LINE TO POINT AT A FILE YOU WANT TO PLAY !!!!
    out.write("add /Users/brian/Desktop/brittlestars.mov\n".getBytes());

    out.write("stop\n".getBytes());
    out.write("info\n".getBytes());
    out.write("get_length\n".getBytes());
    out.write("get_time\n".getBytes());
    out.write("get_title\n".getBytes());
    Thread.sleep(200);
    out.write("is_playing\n".getBytes());
    out.write("play\n".getBytes());
    Thread.sleep(200);
    out.write("is_playing\n".getBytes());
    Thread.sleep(2000);
    out.write("get_time\n".getBytes());
    Thread.sleep(200);
    out.write("get_time\n".getBytes());
    Thread.sleep(200);
    out.write("pause\n".getBytes());
    out.close();
    in.close();
    socket.close();
    System.exit(0);
  }

}



Full List of RC Command

On a Mac, to see the full list of commands available for the rc interface you can do the following:

Start a local rc session in a terminal:
/Applications/VLC.app/Contents/MacOS/VLC --intf rc

Then at the terminal prompt enter longhelp. As of today, the output from this command was:
+----[ CLI commands ]
| add XYZ  . . . . . . . . . . . . . . . . . . . . add XYZ to playlist
| enqueue XYZ  . . . . . . . . . . . . . . . . . queue XYZ to playlist
| playlist . . . . . . . . . . . . .  show items currently in playlist
| search [string]  . .  search for items in playlist (or reset search)
| delete [X] . . . . . . . . . . . . . . . . delete item X in playlist
| move [X][Y]  . . . . . . . . . . . . move item X in playlist after Y
| sort key . . . . . . . . . . . . . . . . . . . . . sort the playlist
| sd [sd]  . . . . . . . . . . . . . show services discovery or toggle
| play . . . . . . . . . . . . . . . . . . . . . . . . . . play stream
| stop . . . . . . . . . . . . . . . . . . . . . . . . . . stop stream
| next . . . . . . . . . . . . . . . . . . . . . .  next playlist item
| prev . . . . . . . . . . . . . . . . . . . .  previous playlist item
| goto, gotoitem . . . . . . . . . . . . . . . . .  goto item at index
| repeat [on|off]  . . . . . . . . . . . . . .  toggle playlist repeat
| loop [on|off]  . . . . . . . . . . . . . . . .  toggle playlist loop
| random [on|off]  . . . . . . . . . . . . . .  toggle playlist random
| clear  . . . . . . . . . . . . . . . . . . . . .  clear the playlist
| status . . . . . . . . . . . . . . . . . . . current playlist status
| title [X]  . . . . . . . . . . . . . . set/get title in current item
| title_n  . . . . . . . . . . . . . . . .  next title in current item
| title_p  . . . . . . . . . . . . . .  previous title in current item
| chapter [X]  . . . . . . . . . . . . set/get chapter in current item
| chapter_n  . . . . . . . . . . . . . .  next chapter in current item
| chapter_p  . . . . . . . . . . . .  previous chapter in current item
|
| seek X . . . . . . . . . . . seek in seconds, for instance `seek 12'
| pause  . . . . . . . . . . . . . . . . . . . . . . . .  toggle pause
| fastforward  . . . . . . . . . . . . . . . . . . set to maximum rate
| rewind . . . . . . . . . . . . . . . . . . . . . set to minimum rate
| faster . . . . . . . . . . . . . . . . . .  faster playing of stream
| slower . . . . . . . . . . . . . . . . . .  slower playing of stream
| normal . . . . . . . . . . . . . . . . . .  normal playing of stream
| rate [playback rate] . . . . . . . . . .  set playback rate to value
| frame  . . . . . . . . . . . . . . . . . . . . . play frame by frame
| fullscreen, f, F [on|off]  . . . . . . . . . . . . toggle fullscreen
| info . . . . . . . . . . . . .  information about the current stream
| stats  . . . . . . . . . . . . . . . .  show statistical information
| get_time . . . . . . . . .  seconds elapsed since stream's beginning
| is_playing . . . . . . . . . . . .  1 if a stream plays, 0 otherwise
| get_title  . . . . . . . . . . . . . the title of the current stream
| get_length . . . . . . . . . . . .  the length of the current stream
|
| volume [X] . . . . . . . . . . . . . . . . . .  set/get audio volume
| volup [X]  . . . . . . . . . . . . . . .  raise audio volume X steps
| voldown [X]  . . . . . . . . . . . . . .  lower audio volume X steps
| achan [X]  . . . . . . . . . . . .  set/get stereo audio output mode
| atrack [X] . . . . . . . . . . . . . . . . . . . set/get audio track
| vtrack [X] . . . . . . . . . . . . . . . . . . . set/get video track
| vratio [X] . . . . . . . . . . . . . . .  set/get video aspect ratio
| vcrop, crop [X]  . . . . . . . . . . . . . . . .  set/get video crop
| vzoom, zoom [X]  . . . . . . . . . . . . . . . .  set/get video zoom
| vdeinterlace [X] . . . . . . . . . . . . . set/get video deinterlace
| vdeinterlace_mode [X]  . . . . . . .  set/get video deinterlace mode
| snapshot . . . . . . . . . . . . . . . . . . . . take video snapshot
| strack [X] . . . . . . . . . . . . . . . . .  set/get subtitle track
| hotkey, key [hotkey name]  . . . . . . . . . . simulate hotkey press
|
| vlm  . . . . . . . . . . . . . . . . . . . . . . . . .  load the VLM
| set [var [value]]  . . . . . . . . . . . . . . . . . set/get env var
| save_env . . . . . . . . . . . .  save env vars (for future clients)
| alias [cmd]  . . . . . . . . . . . . . . . . set/get command aliases
| description  . . . . . . . . . . . . . . . . .  describe this module
| license  . . . . . . . . . . . . . . . . print VLC's license message
| help, ? [pattern]  . . . . . . . . . . . . . . . . .  a help message
| longhelp [pattern] . . . . . . . . . . . . . . a longer help message
| lock . . . . . . . . . . . . . . . . . . . .  lock the telnet prompt
| logout . . . . . . . . . . . . . .  exit (if in a socket connection)
| quit . . . . . . . .  quit VLC (or logout if in a socket connection)
| shutdown . . . . . . . . . . . . . . . . . . . . . . .  shutdown VLC
+----[ end of help ]
Need to generate some JNI headers for files in a Maven project? Below is the simplest snippet I could put together that executes javah. This does nothing fancy, it simply generates headers from a list of files you provide and dumps them into a PROJECT/target/jni directory.

<build>
    <!-- ... -->
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.7</version>
        <executions>
            <execution>
            <phase>process-classes</phase>
            <goals>
                <goal>run</goal>
            </goals>
            <configuration>
                <exportAntProperties>true</exportAntProperties>
                <target>
                   <property name="runtime_classpath" refid="maven.compile.classpath"/>
                   <exec executable="javah">
                       <arg value="-cp"/>
                       <arg value="${runtime_classpath}"/>
                       <arg value="-d"/>
                       <arg value="${project.build.directory}/jni"/>
                       <arg value="org.mbari.terrainnav.jni.TerrainNavNative"/>
                   </exec>
                </target>
            </configuration>
           </execution>
        </executions>
    </plugin>
    <!-- ... -->
</build>

To add your files with native methods just add space separated values to the line that contains org.mbari.terrainnav.jni.TerrainNavNative. For example, if you have classes foo.Bar and foo.Baz then that line would become:

<arg value="foo.Bar foo.Baz"/>


Why did I do it this way? Well, mainly the maven-antrun-plugin provides simple access to Maven's compile classpath. That classpath is required in order to execute javah correctly. I should note that ant has a javah task, but I was unable to get it to work with Maven. On the other hand, getting ant to execute javah directly was trivial.
A version of this post updated for Java 9 is available on Medium.com

First off, let me be perfectly honest, if you have any choice besides using JNI, do that other thing. Dealing with native libraries has been my number one source of pain when supporting Java applications. Now with that warning out of the way, sometimes you just can't avoid using JNI. For those who must go there, I hope this example helps and I salute your bravery and fortitude.

The Java and Scala examples do exactly the same thing. However, the byte code produced is slightly different since Scala doesn't have static methods.

Step 1 - Create a Java or Scala class

Write a Java or Scala class that will interface to the native library:

Java

In Java, methods that will call the native code will use the native keyword in their method signature and have no method body. Create a file named Sample1.java and add the code below:

class Sample1 {

    // --- Native methods
    public native int intMethod(int n);
    public native boolean booleanMethod(boolean bool);
    public native String stringMethod(String text);
    public native int intArrayMethod(int[] intArray);


    // --- Main method to test our native library
    public static void main(String[] args) {
        System.loadLibrary("Sample1");
        Sample1 sample = new Sample1();
        int square = sample.intMethod(5);
        boolean bool = sample.booleanMethod(true);
        String text = sample.stringMethod("java");
        int sum = sample.intArrayMethod(new int[] {1, 1, 2, 3, 5, 8, 13});

        System.out.println("intMethod: " + square);
        System.out.println("booleanMethod: " + bool);
        System.out.println("stringMethod: " + text);
        System.out.println("intArrayMethod: " + sum);
    }
}

The above code can be compiled using:

    javac Sample1.java

Compiling Sample1.java will produce a single class named, Sample1.class

Scala

In Scala, methods that call native code will use the native annotation and, like Java, have no method signature. Create a file named Sample1.scala and add the code below.

class Sample1 {
  // --- Native methods
  @native def intMethod(n: Int): Int
  @native def booleanMethod(b: Boolean): Boolean
  @native def stringMethod(s: String): String
  @native def intArrayMethod(a: Array[Int]): Int
}

// --- Code in App body will get wrapped in a main method on compilation
object Sample1 extends App {

  // --- Main method to test our native library
  System.loadLibrary("Sample1")
  val sample = new Sample1
  val square = sample.intMethod(5)
  val bool = sample.booleanMethod(true)
  val text = sample.stringMethod("java")
  val sum = sample.intArrayMethod(Array(1, 1, 2, 3, 5, 8, 13))

  println(s"intMethod: $square")
  println(s"booleanMethod: $bool")
  println(s"stringMethod: $text")
  println(s"intArrayMethod: $sum")
}

Sample1.scala can be compiled using:

    scalac Sample1.scala

When you compile Sample1.scala you will get three classes: Sample1.class, Sample1$.class, and Sample1$delayedInit$body.class.

Step 2 - Generate a header file with javah

The next step is to have the JVM produce a header file that defines the function prototypes needed for the native implementation.

Java

Creating the native header is really simple for Java. You just need the compiled class, Simple1.class, created in step1. Use the javah program like so:

    javah Sample1

Notice that you do not use the .class extension. javah will produce a file named Sample1.h.

Scala

Creating a native header from Scala code is slightly more involved in that you may need the scala libraries on the javah classpath. In our example, since we are using Scala's App class we definitly need the scala libraries. In this example, we only need scala-library.jar and scala-reflect.jar. I'm working on a Mac and installed Scala via homebrew and my classpath below reflects that. Your path will likely be different depending on your platform.

    SCALA_LIB_HOME=/usr/local/Cellar/scala/2.10.2/libexec/lib/
    SCALA_CP=$SCALA_LIB_HOME/scala-library.jar:$SCALA_LIB_HOME/scala-reflect.jar
    javah -cp $SCALA_CP:. Sample1

Output from javah will be Sample1.h

The Sample1.h file produced here using Scala looks exactly the same as the file produced using the Java class. Just for grins, this is the output of javah:

/
* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class Sample1 */

#ifndef _Included_Sample1
#define _Included_Sample1
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     Sample1
 * Method:    intMethod
 * Signature: (I)I
 */
JNIEXPORT jint JNICALL Java_Sample1_intMethod
  (JNIEnv *, jobject, jint);

/*
 * Class:     Sample1
 * Method:    booleanMethod
 * Signature: (Z)Z
 */
JNIEXPORT jboolean JNICALL Java_Sample1_booleanMethod
  (JNIEnv *, jobject, jboolean);

/*
 * Class:     Sample1
 * Method:    stringMethod
 * Signature: (Ljava/lang/String;)Ljava/lang/String;
 */
JNIEXPORT jstring JNICALL Java_Sample1_stringMethod
  (JNIEnv *, jobject, jstring);

/*
 * Class:     Sample1
 * Method:    intArrayMethod
 * Signature: ([I)I
 */
JNIEXPORT jint JNICALL Java_Sample1_intArrayMethod
  (JNIEnv *, jobject, jintArray);

#ifdef __cplusplus
}
#endif
#endif

Don't you just love C/C++?

Step 3 - Create the native implementation

For both our Scala and Java examples, the native implementation is exactly the same. You can write implementations in either C or C++, although the code will be different. The sample below implements all the methods defined in Sample1.h in C++. Copy and paste the code below into a file named Sample1.cpp:
#include "Sample1.h"
#include <ctype.h>
#include <string.h>

// Mutate array to uppercase
void uppercase(char* str) {
    size_t n = strlen(str);
    for (size_t i = 0; i < n; i++) {
        str[i] = toupper(str[i]); 
    }
}

JNIEXPORT jint JNICALL Java_Sample1_intMethod
(JNIEnv* env, jobject obj, jint num) {
    return num * num;
}


JNIEXPORT jboolean JNICALL Java_Sample1_booleanMethod
(JNIEnv* env, jobject obj, jboolean boolean) {
    return !boolean;
}

JNIEXPORT jstring JNICALL Java_Sample1_stringMethod
(JNIEnv* env, jobject obj, jstring string) {
    const char* str = env->GetStringUTFChars(string, 0);
    char cap[128];
    strcpy(cap, str);
    env->ReleaseStringUTFChars(string, str);
    uppercase(cap);
    return env->NewStringUTF(cap);
}

JNIEXPORT jint JNICALL Java_Sample1_intArrayMethod
(JNIEnv* env, jobject obj, jintArray array) {
    int sum = 0;
    jsize len = env->GetArrayLength(array);
    jint* body = env->GetIntArrayElements(array, 0);
    for (int i = 0; i < len; i++) {
        sum += body[i];
    }
    env->ReleaseIntArrayElements(array, body, 0);
    return sum;
}

Now the fun part, getting this to compile on your particular operating system. Since I'm working on a Mac here's the command line blurb I used to compile Sample1.cpp to a shared native library:

    g++ -dynamiclib -G -Wall -O3 \
        -I/usr/include -I$JAVA_HOME/include -I$JAVA_HOME/include/darwin \
        Sample1.cpp -o libSample1.dylib

A few notes about this:
  1. Notice the -dynamiclib flag, this creates a shared library and is required. On linux use -shared instead. 
  2. You have to include Java's jni headers which are always in at least 2 places. 
    1. $JAVA_HOME/include
    2. $JAVA_HOME/include/<platform> . On Macs that's $JAVA_HOME/include/darwin, on Linux it's $JAVA_HOME/include/linux. I have no idea what it is on Windows.
  3. You may have to set the JAVA_HOME variable on your machine. You can do this on a Mac using: export JAVA_HOME=`/usr/libexec/java_home -v 1.7`
  4. Remember the line in our Java/Scala code that goes: System.loadLibrary("Sample1"). Although the name of the native library is referenced using "Sample1" in our code, it's actually looked up using a 'lib' prefix. So the native library is actually named libSample1.dylib on Mac or libSample1.so on Linux. 
  5. If you're stuck developing for Java 6 on a Mac, change the native libraries extension to .jnilib or Java 6 won't find it. Java 7 on Mac requires .dylib extension. (I did try to warn you that JNI will test your strength)

Step 4 - Running the Code

Running either sample is straight-forward. However, you need to ensure that native library is on Java's or Scala's library path. The simplest way to do this is to add it to the java.library.path when you start your JVM. 

Java

   # Assumes all files are in the same directory.
    # If they are not, replace $(pwd) with the directory containing libSample1.so or libSample1.dylib
    java -Djava.library.path=$(pwd) -cp . Sample1

Scala

    scala -Djava.library.path=$(pwd) -cp . Sample1

or 

    SCALA_LIB_HOME=/usr/local/Cellar/scala/2.10.2/libexec/lib/
    SCALA_CP=$SCALA_LIB_HOME/scala-library.jar:$SCALA_LIB_HOME/scala-reflect.jar
    java -Djava.library.path=$(pwd) -cp $SCALA_CP:. Sample1

A Note about Deployment

It's not always feasible to set the java.library.path variable (e.g. when using Java Web Start). However, it is not particularly difficult to hack the path used internally by Java to add shared libraries at runtime. This is, however, a topic beyond the scope of this posting.
Sublime Text is a great text editor. Here's a little AppleScript that can help open folders or files selected in the finder directly with Sublime Text 3. To use the script:


  1.  Open AppleScriptEditor 
  2.  Open a new editor (File -> New) 
  3. Paste the code below into the editor 
  4. Press the compile button to make sure it was copied correctly 
  5. Export as an Application (File -> Export, Make sure you select 'Application' as the File Format. I usually save my scripts to '~/Library/Scripts' but the location isn't so important. 
  6. Drag your newly saved application onto the Finder's toolbar 
  7. That's it. Just press it whenever you want to open selected Files/Folders in Sublime Text 3 
  8. You can change the icon on your new Application by following these instructions.

(*
 Open in Sublime Text
 To use:
  * Drag Open In TextMate to the toolbar of any finder
  window to add it to the toolbar
*)

on run
 tell application "Finder"
  if selection is {} then
   set finderSelection to folder of the front window as string
  else
   set finderSelection to selection as alias list
  end if
 end tell
 
 subl(finderSelection)
end run

-- script was drag-and-dropped onto
on open (theList)
 subl(theList)
end open

-- open in Sublime Text
on subl(listOfAliases)
 tell application "Sublime Text"
  activate
  open listOfAliases
 end tell
end subl
If you've done any work with Scala, you quickly realize that the compiler can be rather, ahem, slow. (We're talking C++ compiler slow). Now, I'm not knocking the scala compiler, I realize it's doing a lot of work, so I forgive it. But still, I just want things to compile faster. Typesafe pushes sbt as the solution to slow compilation as it supports incremental compilation and reduced startup time (i.e You can launch sbt and keep it running; you don't have to spin up a JVM for every compile cycle.). However, I think Typesafe has finally realized that Maven is very popular as a Scala build tool and so have released zinc, which is just the standalone version of sbt's incremental compiler. I've found that combining zinc with mvnsh results in much, much, much faster compilation cycles than just plain Maven. Here's how to get it going ... oh, the instructions are for a Mac, but you'll get the gist of it.

Download and Install Zinc

If you have homebrew, run the following:
brew install zinc
Otherwise:
  1. Download zinc from https://github.com/typesafehub/zinc
  2. Extract the zip file
  3. cd into zinc-[version]/bin
  4. Start zinc from a terminal with zinc -start
  5. Note: when you're done with zinc you'll need to stop it with zinc -shutdown

Install mvnsh

Homebrew is again your friend ... just run the following:
brew install maven-shell
Otherwise, you can get the source and build mvnsh from https://github.com/jdillon/mvnsh

Configure your Maven project to use Zinc

At this point, I'm assuming you already have a pom.xml configured for building a scala project. If not you can just use mvn archetype:generate and select the 'simple scala project' template. That will give you a pom.xml file to work with. Make sure that the pom file is using the very excellent scala-maven-plugin; and tell the plugin to use zinc if it's available. So your pom include something like the snippet below ... Notice the line that has useZincServer; that's important to have:
<build>
    <plugins>
        <plugin>
            <groupId>net.alchim31.maven</groupId>
            <artifactId>scala-maven-plugin</artifactId>
            <version>3.1.0</version>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>testCompile</goal>
                    </goals>
                    <configuration>
                        <args>
                            <arg>-make:transitive</arg>
                            <arg>-dependencyfile</arg>
                            <arg>${project.build.directory}/.scala_dependencies</arg>
                        </args>
                    </configuration>
                </execution>
                <execution>
                    <id>scala-test-compile</id>
                    <phase>process-test-resources</phase>
                    <goals>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <jvmArgs>
                    <jvmArg>-Xss2m</jvmArg>
                </jvmArgs>
                <recompileMode>incremental</recompileMode>
                <useZincServer>true</useZincServer>
            </configuration>
        </plugin>
    </plugins>
</build>

Building your Project

  1. Start zinc in a terminal (zinc -start)
  2. Start mvnsh in a terminal (mvnsh)
  3. In mvnsh, issue your build commands (e.g. mvn package)
Any builds from scratch (first builds, builds after 'mvn clean') will still take a relatively long time (coffee break!). But subsesquent builds will be much, much faster. If you forget to start zinc, no worries, maven will fall back to using the standard compiler.
I normally use IntelliJ IDEA and it's excellent Scala plugin for Scala development*. However, IntelliJ gave me a truly bogus autocomplete suggestion while adding a ChangeListener to a JavaFX Property referencing a Node:

Does Not Compile

What IntelliJ giveth:
nodeProperty.addListener(new ChangeListener[_ >: Node] {
  def changed(p1: ObservableValue[_ <: _ >: Node], p2: _ >: Node, p3: _ >: Node) {}
})

Compiles

I had to change it to the following:
nodeProperty.addListener(new ChangeListener[Node] {
  def changed(p1: ObservableValue[_ <: Node], p2: Node, p3: Node) {}
})
* A footnote about developing with Scala. I normally build projects with Maven. The very excellent scala-maven-plugin, when used along-side zinc and mvnsh make for very quick code-compile cycles.

I also sometimes use SBT/ensime/Sublime Text. However, SBT and I usually agree to disagree. Fortunately, for many projects creating a dual Maven/SBT build is relatively straightforward.