Komputation

Komputation is a neural network framework for the Java Virtual Machine written in Kotlin and CUDA C.

Maven

Komputation is available through Maven Central:

<dependency>
    <groupId>com.komputation</groupId>
    <artifactId>komputation</artifactId>
    <version>0.12.5</version>
</dependency>

Layers

• Entry points:
  • Input
  • Lookup
• Standard feed-forward networks:
  • Weighting
  • Bias
  • Projection
  • Dense
• Convolutional neural networks (CNNs):
  • Convolution
  • Max-pooling
• Recurrent neural networks:
  • Recurrent layer
  • Bidirectional recurrent layer

• Dropout

• Activation functions:
  • Identity
  • Rectified Linear Units (ReLUs)
  • Sigmoid
  • Softmax
  • Tanh
• Other layers:
  • Stack
  • Exponentiation
  • Normalization

CPU demos

• Boolean functions:
  • AND
  • NOT
  • XOR
• Total:
  • Fixed length
  • Variable length
• Running total:
  • Left-to-right:
    • Fixed length
    • Variable length
  • Right-to-left:
    • Fixed length
    • Variable length
  • Bidirectional:
    • Fixed length
    • Variable length
• Increment:
  • One layer
  • Two layers
• Word embedding toy problem:
  • Feed-forward network
  • CNN with one filter width
  • CNN with two filter widths

• Sequence labeling toy problem

• Computer vision toy problem

• MNIST:
  • Minimal
  • Dropout
• TREC:
  • One filter width
  • Two filter widths

GPU/CUDA demos

• Boolean functions:
  • AND
  • Negation
  • XOR
• Word embedding toy problem:
  • Feed-forward network
  • CNN with one filter width
  • CNN with two filter widths
• Total:
  • Fixed length
• Increment:
  • One layer
  • Two layers
• MNIST:
  • Minimal
  • Dropout
• TREC:
  • One filter width
  • Two filter widths

Sample code

The following code instantiates a GPU-accelerated convolutional neural network for sentence classification:

    val sentenceClassifier = cudaNetwork(
        batchSize,
        lookup(embeddings, maximumDocumentLength, embeddingDimension, optimization),
        convolution(numberFilters, filterWidth, filterHeight, initialization, optimization),
        relu(),
        dropout(random, keepProbability),
        dense(numberCategories, Activation.Softmax, initialization, optimization)
    )

See the TREC demo for more details.

Initialization

• Provided
• Constant
• Gaussian
• He
• Identity
• Uniform
• Zero

Loss functions

• Cross-entropy loss
• Logistic loss
• Squared loss

Optimization

• Stochastic Gradient Descent
• Historical:
  • Momentum
  • Nesterov’s Accelerated Gradient
• Adaptive:
  • Adagrad
  • Adadelta
  • RMSProp
  • Adam