CSF Toolkit Quick start guide. This is trial version with compiled Python code (P-code) that can reproduce published results. They were created using Python version 3.7.11, and require this version or similar to run. Source code including GPU version is available, contact author. CSF Toolkit is based on Theano. A version that runs under Aesara is also provided. Note: csftk.py is a compiled python program, but it can be run like a python script: python csftk.py (You also need all the .pyc files) Directory contents: files/*.py* CSF Toolkit python program files/*.mod Model definition files The model files are included: ar.mod : auto-regressve model mel.mod : mel-cepstrum model sinar.mod : sine-wave + ar noise lev.mod : auto-regressve model that first computes auto-correlation Paul 02 Apr 2022 Installation guide is below. ------------------------------------- What is CSF Toolkit? CSF Toolkit is a graphical user interface (GUI) that implements feature extraction according to the class-specific feature (CSF) method, which is based on PDF projection. When calculating CSF features, a correction factor (the J-function) is computed to correct likelihood functions so that they are referenced to the input data. For example. let x be an input data vector and let z1 and z2 be two different feature vectors, both extracted from x. Let p(z1) and p(z2) be probability density function (PDF) estimates for the two features. Normally, comparing log p(z1) and log p(z2) is meaningless. Just by scaling one featuzre, the PDF will change. However, comparing (J1 + log p(z1)) and (J2 + log p(z2)) *is* meaningful because these two quantities are actiuşalyl PDF estimates of x. For more information on CSF, see http://class-specific.com/csf/ The CSF Toolkit extracts features and their J-functions. In addition, it allows reconstruction of the input data from any feature. The quality of the reconstruction is a qualitative measure of the goodness of a feature. In CST Toolkit, you can specify models (files with ending .mod) consisting of a cascade of layers. Each layer has an inverse function to re-construct the input. For example, the inverse function for 'arx', which computes the autoregressice coefficients, is filtered white noise that fits the given AR model. The following layers are provided: hann3 Hanning-weighted and 2/3 overlapped segmentation. Follows: Is always first layer Parameters: N (FFT size , divisible by 3). Output dimension: NFFT fftsq FFT magnitide squared. Follows: 'hann3' Parameters: None Output dimension: NFFT/2+1 arx Auto-regressive (AR) coefficients. Follows: 'hann3' Parameters: P (AR model order). Output dimension: P+1 rc Reflection coefficients (autoregressive model). Follows: 'arx' or 'lev'. Parameters: None Output dimension: (same as input: P+1) bilin for log bilinear transformations (log-area ratio, or LAR) coefficients. This is a 1:1 transformation applied to reflection coefficients (rc). Follows: 'rc' Parameters: None Output dimension: (same as input: P+1) melbnk Bank of MEL spectral bands. Normally applied to output of 'fftsq'. Follows: 'fftsq' Parameters: k (number of bands), typ (band type) # typ=0: MEL spacing, Hanning shape # typ=1: MEL spacing, triangular shape # typ=2: LIN spacing, Hanning shape # typ=3: LIN spacing, triangular shape # typ=4: ACF matrix - produces auto-correlation function (ACF) # of order k-1 # typ=5: MEL spacing, rectangular bands (much faster to compute) # typ=6: LIN spacing, rectangular bands (much faster to compute) Output dimension: k log Takes log Follows: 'fftsq' or 'melbnk' Parameters: None Output dimension: (same as input) lev Levinson algorithm. Converts ACF to AR coefs. Follows: 'melbnk' with typ=4 Parameters: None Output dimension: (same as input) sinar Sine-wave plus auto-regressive noise. Parameters: P (AR model order), k (number of sine-waves), dc (if set to 1, also computes DC value) Follows: 'hann3' Output dimension: (3k+P+1+dc): k sinewave frequencies, 2k amplitudes (real+imag), P+1 AR coefs, plus the dc level (if dc=1). (currently only dc=0 is supported) c2r Cartesian-to-radius. Converts the 2k amplitudes (real+imag) from 'sinar' into k amplitudes. Follows: 'sinar' Output dimension: (2k+P+1+dc) dct Discrete cosine transform. Follows: 'melbnk' Output dimension: (same as input) sqrt Square root. Follows: 'fftsq' Output dimension: (same as input) ------- pre-specified models ----- The following modules are available in the files directory: ar.mod : auto-regressve model hann3 --> arx --> rc --> bilin mel.mod : mel-cepstrum model hann3 --> fftsq --> melbnk --> log sinar.mod : sine-wave + ar noise hann3 --> sinar --> c2r lev.mod : auto-regressve model hann3 --> fftsq --> melbnk(4) --> lev --> rc --> bilin ------------------------------------------------------ Installation guide The toolkits are based on Theano and also work with Aesara which is a newer version of Theano, but has limited GPU support. It is best to use Linux, but it also works on Windows. I've tested them on Windows and provide the following installation guide. Installing Theano and CSF or PBN Toolkits on Windows. #----- WINDOWS 7 ------------------ 1. Install Miniconda-3 from: http://repo.anaconda.com/miniconda If you have an older version of Windows, you might need an older version of Miniconda3. For Windows 7, I had to go back to version 4.1.2 2. After installing Miniconda3, run the Miniconda-3 Prompt by navigating from the system start menu: System Start Menu -> All Programs -> Anaconda3 -> Anaconda Prompt (Miniconda3) 3. In the Miniconda command window, type: > conda install theano > conda install matplotlib > conda install future > python -m pip install sklearn > python -m pip install playsound > python -m pip install pygame #----- WINDOWS 10 ------------------ Update: a user has reported the following successful installation in Windows 10: 1. Install conda=4.12.0 from the source reported for Windows 7 2. create a conda environment for the needed python version, since otherwise the magic numbers do not match: conda create -n your_env python=3.7.0 3. Install the following packages into the created environment: conda install numpy scipy theano conda install tk conda install future conda install packaging conda install matplotlib python -m pip install sklearn python -m pip install playsound pip install pygame #----- Finishing the install ------------------ 4. For some reason, installing Theano there is a file missing: "scan_perform.c" I have included this file in the distribution. Go to directory scan_module/ the theano installation directory. For me it was C:/Users/PaulB/Miniconda3/Lib/site-packages/theano/scan_module/ Add a sub-directory called 'c_code' and copy the file scan_perform.c into it. In other words: C:/Users/PaulB/Miniconda3/Lib/site-packages/theano/scan_module/c_code/scan_perform.c Your code will run faster with this file. 5. In the Miniconda3 command window, run: > set THEANO_FLAGS=mode=FAST_RUN,device=cpu,floatX=float64 You can also specify 'float32' which runs faster. This environment variable can also be set permanently using system settings. To run the toolkit: > python csftk.py It may take a long time to compile functions the first time (took 20 minutes for me). I may have missed some Python packages. If an error message appears about a missing Python package, try the 'conda' or 'pip' installation as shown above. Using with AESARA Aesara is recommended if you do not have a GPU 1. First download the aesara version of the code. 2. In the Miniconda command window, type: > python -m pip install aesara > set PBNTK_BACKEND=AESARA > set AESARA_FLAGS=mode=FAST_RUN,device=cpu,floatX=float64 (or float32) These environment variable can also be set permanently using system settings. > python csftk.py p.m.baggenstoss@ieee.org --------------- Quick-start guide ---------- 0. Download all the software from http://class-specific.com/csftk/ 1. Create a data directory in the current working directory with the sub-folders ./data/events/ ./data/features/ 2. To create a data set, for example "cup" a. create folder ./data/events/cup/ b. create the text file ./data/events/cup/event_list.txt This file has one line per event. Each line is form : (file name) (channel number) (sample rate) (start time in sec) (end time in sec): Example : ------------------ /media/paul.baggenstoss/Transcend/DCASE/fan/train/event1.wav 0 16000 0 10 /media/paul.baggenstoss/Transcend/DCASE/fan/train/event2.wav 0 16000 0 10 /media/paul.baggenstoss/Transcend/DCASE/fan/train/event3.wav 0 16000 0 10 : : ------------------ You can also put all your .wav files in ./data/events/cup/, in which case, you don't need full path names: ------------------ event1.wav 0 16000 0 10 event2.wav 0 16000 0 10 event3.wav 0 16000 0 10 : : ------------------ You can specify more than one event for a given .wav file if you use different start and end times I have uploaded the archives 'quart.tgz' and 'cup.tgz', which are examples of event data directories. This data is explained in http://class-specific.com/os/ You should un-pack these archives and set up the directories data/events/cup data/events/quart 3. Calculate features using csf_toolkit: 1. python csf_tk.py 2. In DATA field, select data class '0', and enter 'cup', then press return 3. press 'SCAN', the toolkit should print out "Found XXX events in data set cup" 4. Select MODEL 0 in drop-down menu. Enter 'ar' in the model name field. Press LOAD (need file ar.mod) 5. You can change FFT size or add layers, then press 'APPLY' If you made changes, you can save them to 'ar.mod' by pressing SAVE 6. Compile the feature function: press 'COMP' 7. Enable 'Plot' checkbox. 8 selec an event number ('ev') 9 Press CALC to calculate and display features. 10 You can see the original and reconstructed spectrograms and play the original and re-constructed sound using 'PLAY', 'PLAY SYN' 11 If you are happy with the results, calculate and save the features by checking 'Write' and un-check 'Plot', then press CALC again. 4. Note for Windows. I found that on Windows, compiling took a long time. This is because I have taken advantage of the C-modules that are C-code versions of functions that run *much* faster but are slower to compile the first time. If you compile a function again a second session, it will be faster than the first time. If you compile a function a second time in one session, it goes very fast. To avoid unneccessary waiting, always assign a different model number to a new model using the MODEL drop-down menu. That way, you can return to an already-compiled module using the drop-down menu without needing to re-compile it. 5. Speeding up. If you are not using CSF Toolkt to compute the J-functon ( you just need the features), you can un-check ``SPA", to avoid SPA calculation and speed up computations. If you do need the J-function, and are using ``melbnk", use types 5 and 6 (rectangular bands) to greatly speed up calculations.