SNNS - Stuttgart Neural Network Simulator

User Manual, Version 4.0

Andreas Zell, Günter Mamier, Michael Vogt, Niels Mache, Ralf Hübner, Sven Döring, Kai-Uwe Herrmann, Tobias Soyez, Michael Schmalzl, Tilman Sommer, Artemis Hatzigeorgiou, Dietmar Posselt, Tobias Schreiner, Bernward Kett, Gianfranco Clemente

external contributions by

Martin Reczko, Martin Riedmiller, Mark Seemann, Marcus Ritt, Jamie DeCoster, Jochen Biedermann, Joachim Danz, Christian Wehrfritz

SNNS
Stuttgart Neural Network Simulator
User Manual, Version 4.0
Report No. 6/95

All Rights reserved

• Contents
• Introduction to SNNS
• Licensing, Copying and Acknowledgments
  • SNNS License
  • How to obtain SNNS
  • Installation
  • Acknowledgments
  • New Features of Release 4.0
• Neural Network Terminology
  • Building Blocks of Neural Nets
    • Units
    • Connections (Links)
    • Sites
  • Update Modes
  • Learning in Neural Nets
  • Generalization of Neural Networks
  • An Example of a simple Network
• Using the Graphical User Interface
  • XGUI Files
  • Windows of XGUI
    • Shell window
    • Confirmer
    • Manager Panel
    • Info Panel
    • 2D Displays
    • Setup Panel
    • Unit Function Displays
    • File Browser
      • Loading and Saving Networks
      • Loading and Saving Patterns
      • Loading and Saving Configurations
      • Saving a Result file
      • Defining the Log File
    • Help Windows
    • Print Panel
    • Remote Panel
    • Weight Display
    • Graph Window
    • Projection Panel
  • Parameters of the Learning Functions
  • Update Functions
    • ART1_Stable
    • ART1_Synchronous
    • ART2_Stable
    • ART2_Synchronous
    • ARTMAP_Stable
    • ARTMAP_Synchronous
    • Auto_Synchronous
    • BAM_Order
    • BBTT_Order
    • CC_Order
    • CounterPropagation
    • Dynamic_LVQ
    • Hopfield_Fixed_Act
    • Hopfield_Synchronous
    • JE_Order
    • JE_Special
    • Kohonen_Order
    • RCC_Order
    • Random_Order
    • Random_Permutation
    • Serial_Order
    • Synchronous_Order
    • TimeDelay_Order
    • Topological_Order
  • Initialization Functions
    • ART1_Weights
    • ART2_Weights
    • ARTMAP_Weights
    • CC_Weights
    • ClippHebb
    • CPN_Weights
    • DLVQ_Weights
    • Hebb
    • Hebb_FixAct
    • JE_Weights
    • Kohonen_Weights
    • PseudoInv
    • Randomize_Weights
    • Random_Weights_Perc
    • RBF_Weights
    • RBF_Weights_Kohonen
    • RBF_Weights_Redo
    • RCC_Weights
    • RM_Random_Weights
  • Creating and Editing Unit Prototypes and Sites
• Handling Patterns with SNNS
  • Handling Pattern Sets
  • Fixed Size Patterns
  • Variable Size Patterns
• Graphical Network Editor
  • Editor Modes
  • Selection
    • Selection of Units
    • Selection of Links
  • Use of the Mouse
  • Short Command Reference
  • Editor Commands
  • Example Dialogue
• Network Creation Tools
  • BigNet for Feed-Forward Networks
    • Terminology of the Tool BigNet
    • Buttons of BigNet
    • Plane Editor
    • Link Editor
      • Example 1: Receptive Fields in Two Dimensions
      • Example 2: Moving in Different Dimensions
    • Create Net
  • BigNet for Time-Delay Networks
    • Terminology of Time-Delay BigNet
    • Plane Editor
    • Link Editor
  • BigNet for ART-Networks
  • BigNet for Self-Organizing Maps
  • BigNet for Autoassociative Memory Networks
  • BigNet for Partial Recurrent Networks
    • BigNet for Jordan Networks
    • BigNet for Elman Networks
• Network Analyzing Tools
  • Inversion
    • The Algorithm
    • Inversion Display
    • Example Session
  • Network Analyzer
    • The Network Analyzer Setup
    • The Display Control Window of the Network Analyzer
• Neural Network Models and Functions
  • Backpropagation Networks
    • Vanilla Backpropagation
    • Enhanced Backpropagation
    • Batch Backpropagation
    • Backpropagation with Weight Decay
  • Quickprop
  • RPROP
    • Changes in Release 3.3
    • General Description
    • Parameters
  • Backpercolation
  • Counterpropagation
    • Fundamentals
    • Initializing Counterpropagation
    • Counterpropagation Implementation in SNNS
  • Dynamic Learning Vector Quantization (DLVQ)
    • DLVQ Fundamentals
    • DLVQ in SNNS
    • Remarks
  • Backpropagation Through Time (BPTT)
  • The Cascade Correlation Algorithms
    • Cascade-Correlation (CC)
      • The Algorithm
      • Mathematical Background
    • Pruned-Cascade-Correlation (PCC)
      • The Algorithm
      • Mathematical Background
    • Recurrent Cascade-Correlation (RCC)
      • The Algorithm
      • Mathematical Background
    • Using the Cascade Algorithms in SNNS
  • Time Delay Networks (TDNNs)
    • TDNN Fundamentals
    • TDNN Implementation in SNNS
      • Activation Function
      • Update Function
      • Learning Function
    • Building and Using a Time Delay Network
  • Radial Basis Functions (RBFs)
    • RBF Fundamentals
    • RBF Implementation in SNNS
      • Activation Functions
      • Initialization Functions
      • Learning Functions
    • Building a Radial Basis Function Application
  • ART Models in SNNS
    • ART1
      • Structure of an ART1 Network
      • Using ART1 Networks in SNNS
    • ART2
      • Structure of an ART2 Network
      • Using ART2 Networks in SNNS
    • ARTMAP
      • Structure of an ARTMAP Network
      • Using ARTMAP Networks in SNNS
    • Topology of ART Networks in SNNS
  • Self-Organizing Maps (SOMs)
    • SOM Fundamentals
    • SOM Implementation in SNNS
      • The KOHONEN Learning Function
      • The Kohonen Update Function
      • The Kohonen Init Function
      • The Kohonen Activation Functions
      • Building and Training Self-Organizing Maps
      • Evaluation Tools for SOMs
  • Autoassociative Networks
    • General Characteristics
    • Layout of Autoassociative Networks
    • Hebbian Learning
    • McClelland & Rumelhart's Delta Rule
  • Partial Recurrent Networks
    • Models of Partial Recurrent Networks
      • Jordan Networks
      • Elman Networks
      • Extended Hierarchical Elman Networks
    • Working with Partial Recurrent Networks
      • The Initialization Function JE_Weights
      • Learning Functions
      • Update Functions
  • Stochastic Learning Functions
    • Monte-Carlo
    • Simulated Annealing
• Pruning Algorithms
  • Background of Pruning Algorithms
  • Theory of the implemented algorithms
    • Magnitude Based Pruning
    • Optimal Brain Damage
    • Optimal Brain Surgeon
    • Skeletonization
    • Non-contributing Units
  • Pruning Nets in SNNS
• 3D-Visualization of Neural Networks
  • Overview of the 3D Network Visualization
  • Use of the 3D-Interface
    • Structure of the 3D-Interface
    • Calling and Leaving the 3D Interface
    • Creating a 3D-Network
      • Concepts
      • Assigning a new z-Coordinate
      • Moving a z-Plane
      • Displaying the z-Coordinates
      • Example Dialogue to Create a 3D-Network
    • 3D-Control Panel
      • Transformation Panels
      • Setup Panel
      • Model Panel
      • Project Panel
      • Light Panel
      • Unit Panel
      • Links Panel
      • Reset Button
      • Freeze Button
    • 3D-Display Window
• Running SNNS as Batch Job
  • The Snnsbat Environment
  • Using Snnsbat
  • Calling Snnsbat
• Tools for SNNS
  • Overview
  • Analyze
    • Analyzing Functions
  • Bignet
  • Convert2snns
    • Setup and Structure of a Control, Weight, Pattern File
  • Feedback-gennet
  • Mkhead
  • Mkout
  • Mkpat
  • Netlearn
  • Netperf
  • Pat_sel
  • Snns2c
    • Program flow
    • Including the Compiled Network in the Own Application
      • Interfaces:
      • Example:
    • Special Network Architectures
      • TDNN:
      • CPN:
      • DLVQ:
      • BPTT:
    • Activation Functions
      • Supported Activation Functions:
      • Including Own Activation Functions:
    • Error messages
• Design of the Simulator Kernel
  • Network Model of the Simulator
  • Design Factors
  • Layer Model of the Simulator Kernel
• Internal Data Structures
  • Unit Array
  • Sites
  • Links
  • Network Memory Management
    • Link/Site Arrays
    • Symbol Table
  • Unit Flags
  • Function Table
• Kernel Function Interface
  • Overview
  • Unit Functions
    • Unit Enquiry and Manipulation Functions
    • Unit Definition Functions
  • Site Functions
    • Functions for the Definition of Sites
    • Functions for the Manipulation of Sites
  • Link Functions
  • Functions for the Manipulation of Prototypes
  • Functions to Read the Function Table
  • Network Initialization Functions
  • Functions for Activation Propagation in the Network
  • Learning and Pruning Functions
  • Functions for the Manipulation of Patterns
  • File I/O Functions
  • Functions to Search the Symbol Table
  • Miscelaneous other Interface Functions
  • Memory Management Functions
  • ART Interface Functions
  • Error Messages of the Simulator Kernel
• Transfer Functions
  • Predefined Transfer Functions
  • User Defined Transfer Functions
• Simulator Kernel Implementation
• Implementation of the User Interface
  • Administration of the Windows
  • Main Program
  • Manager Panel
  • Layer Panel
  • Graphic Windows
    • Event Handler for Mouse and Window Events
    • Event Handler for Keyboard Events
    • Editor Actions
    • Setup Panel
    • Freezing Displays
  • List Module
  • File Panel
  • Help Window
  • Confirmer
  • Graphic
  • Selection Mechanism
  • Interface to the Simulator Kernel
• 3D-Display Implementation
  • Contents of the Modules
  • Global Data Types and Variables
  • Drawing the Network in 3D
  • Low Level Drawing Routines
  • Matrix Calculations
  • The 3D Display Window
  • Panels
• Kernel File Interface
  • The ASCII Network File Format
  • Form of the Network File Entries
  • Grammar of the Network Files
    • Conventions
      • Lexical Elements of the Grammar
      • Definition of the Grammar
    • Terminal Symbols
    • Grammar:
  • Grammar of the Pattern Files
    • Terminal Symbols
    • Grammar
• Example Network Files
  • Example 1:
  • Example 2:
• Example Snnsbat Protocol File
• References
• Index
• About this document ...