Difference between revisions of "Software"

Revision as of 11:06, 1 October 2018 (view source) Hfrancot (Talk | contribs) (→‎Testing OpenNI + NiTE) ← Older edit		Revision as of 11:07, 1 October 2018 (view source) Hfrancot (Talk | contribs) (→‎Testing OpenNI + NiTE) Newer edit →
Line 96:		Line 96:
	./NiUserTracker		./NiUserTracker
−	[[File:NiTE_example.png|200px|left|NiUserTracker example running on Ubuntu 18.04]]	+	[[File:NiTE_example.png|400px|NiUserTracker example running on Ubuntu 18.04]]
	=== OpenKinect drivers===		=== OpenKinect drivers===

---

Revision as of 11:07, 1 October 2018

This section is related to all those requeriments and instructions needed either for develeopment or only the execution of software. Since this project is aimed to work in several operative systems beyond Microsoft Windows such as UNIX platforms, we are using open-source tools such as libfreenect, openCV and openFrameWorks that allow communication with Kinect and implementation of computer vision algorithms.

Contents 1 Sofware prerequisites needed for development and execution in C++ 1.1 OpenNI drivers 1.1.1 Check OpenNI dependencies 1.1.2 Install OpenNI drivers 1.1.3 Install NiTE Middleware (for skeleton model) 1.1.4 Testing OpenNI + NiTE 1.2 OpenKinect drivers 1.2.1 Installing OpenKinect drivers 1.3 OpenCV libraries 1.3.1 Installing OpenCV libraries 1.4 Packages dependencies 1.4.1 For OpenNI + NiTE 1.4.2 For libfreenect 1.4.3 For OpenCV 1.5 Flags needed for development in C++ 2 Using openFrameworks as toolkit for development in C++ 2.1 Downloading openFrameworks 2.2 Prerequisites for communication wih Kinect 2.3 Generating a new openFrameworks project 3 Downloading and running project 3.1 Downloading project from git repository 3.2 Compiling 3.3 Executing

Sofware prerequisites needed for development and execution in C++

OpenNI drivers

OpenNI is a non-profit consortium formed to promote and standardize the compatibility and interoperability of Natural Interaction (NI), devices, applications and middleware. Today, OpenNI is the largest 3D sensing development framework and community. Its open source SDK is the recognized standard for developing computer vision middleware and 3D solutions.

Check OpenNI dependencies

Requirements:

1. GCC 4.x:
   • From: http://gcc.gnu.org/releases.html
   • Or via apt: sudo apt-get install g++
2. Python 2.6+/3.x
   • From: http://www.python.org/download/
   • Or via apt: sudo apt-get install python
3. LibUSB 1.0.x
   • From: http://sourceforge.net/projects/libusb/files/libusb-1.0/
   • Or via apt: sudo apt-get install libusb-1.0-0-dev
4. FreeGLUT3:
   • From: http://freeglut.sourceforge.net/index.php#download
   • Or via apt: sudo apt-get install freeglut3-dev
5. JDK 8.0:
   • From: https://www.oracle.com/technetwork/java/javase/documentation/jdk8-doc-downloads-2133158.html
   • Or via apt: sudo apt-get install openjdk-8-jdk
6. Doxygen:
   • From: http://www.stack.nl/~dimitri/doxygen/download.html#latestsrc
   • Or via apt: sudo apt-get install doxygen
7. Graphviz
   • From: http://www.graphviz.org/Download_windows.php
   • Or via apt: sudo apt-get install graphviz

Install OpenNI drivers

This installation guide show how to install PrimeSense drivers for Kinect in Linux Ubuntu distro. However should work for most of Linux distributions.

Taken from this guide

• Install OpenNI

 #Create a 'kinect' folder in your home directory to store installation source of drivers:
mkdir -p ~/kinect 
cd ~/kinect

#Download from the OpenNI modules. Make sure you get the 'unstable' branch: 
git clone https://github.com/gdsosar/OpenNI.git 
cd OpenNI
git checkout develop
cd Platform/Linux/CreateRedist
./RedistMaker
 
#If that succeeds, there a Redist folder should be created with an install script. Run it: 
cd ~kinect/OpenNI/Platform/Linux/Redist/OpenNI-Bin-Dev-Linux-x64-v1.5.8.5/
sudo ./install.sh

• Install OpenNI 2.2

#Download OpenNI v2.2  from http://hpclab.ucentral.edu.co/~gsosar/packages/OpenNI-Linux-x64-2.2.0.33.tar.bz2 into ~/kinect.
cd ~/kinect
tar -jxvf OpenNI-Linux-x64-2.2.0.33.tar.bz2
cd OpenNI-Linux-x64-2.2
sudo ./install.sh

• Install PrimeSense drivers

#many blogs recommend installing from https://github.com/avin2/SensorKinect.git 
#This didn't work for me, but the following the following repo from ph4m worked: 
cd ~/kinect
git clone https://github.com/ph4m/SensorKinect.git
#make sure you checkout the 'unstable' branch 
git checkout unstable 

#Install it 
cd SensorKinect/Platform/Linux/CreateRedist
./RedistMaker

#If that succeeds, there a Redist folder should be created with an install script. Run it: 
cd ~/kinect/SensorKinect/Platform/Linux/Redist/Sensor-Bin-Linux-x64-v5.1.2.1/
sudo ./install.sh

Install NiTE Middleware (for skeleton model)

The PrimeSense NiTE is an advanced and robust 3D computer vision middleware available today, boasting a thin host, minimal CPU load, and multiplatform support

Taken from this guide

• Install NiTE

#Download NiTE v1.5.2.23 from http://hpclab.ucentral.edu.co/~gsosar/packages/NITE-Bin-Linux-x64-v1.5.2.23.tar.bz2 into ~/kinect.
#The latest NITE SDK from  http://openni.ru/files/nite/index.html may be fine as well.
cd ~/kinect
tar -jxvf NITE-Bin-Linux-x64-v1.5.2.23.tar.bz2
cd NITE-Bin-Dev-Linux-x64-v1.5.2.23
sudo ./install.sh

Testing OpenNI + NiTE

The NiUserTracker example allows you to visualize the Kinect depth map detection, as well as the human segmentation and skeleton tracking function with NiTE. Just follow these instructions to compile and run the sample:

cd ~/kinect
cd OpenNI/Samples/NiUserTracker
g++ main.cpp opengles.cpp SceneDrawer.cpp -o NiUserTracker -lGL -lglut -lOpenNI -I /usr/include/ni
./NiUserTracker

OpenKinect drivers

OpenKinect is an open community whose purpose is to release open-source programming tools and drivers to make use of Kinect hardware. Current libraries enable Kinect for use in Windows, UNIX and Mac platforms.

Installing OpenKinect drivers

OpenKinect package is named libfreenect. Detailed information about installation is here

• Ubuntu official package installation

Starting from Ubuntu 11.10 (Oneiric) and Debian 7 (Wheezy), Ubuntu and Debian provide official packages of libfreenect. You can install them easily in a console:

$ sudo apt-get install freenect

In Ubuntu 12.04 the gspca kernel driver prevent libfreenect from claiming the Kinect device in user-mode. Either remove and blacklist the module

$ sudo modprobe -r gspca_kinect

$ sudo modprobe -r gspca_main

$ echo "blacklist gspca_kinect" |sudo tee -a /etc/modprobe.d/blacklist.conf

or install a recent versions of libfreenect that can automatically detach the kernel driver by adding Florian Echtler libtisch PPA (see below) with updated libfreenect packages for Ubuntu 12.04.

The freenect device is accessible to any user belonging to the group 'plugdev'. By default, a desktop user belongs to the plugdev group but if you need to add him to the group:

$ sudo adduser $USER plugdev

then log out and log in again.

• Ubuntu manual installation

Quick copy-paste instructions to get up-and-running instantly:

git clone git://github.com/OpenKinect/libfreenect.git
cd libfreenect
mkdir build
cd build
cmake ..
make
sudo make install
sudo ldconfig /usr/local/lib64/
sudo glview

To use Kinect as a non-root user do the following:

sudo adduser $USER video

Also make a file with rules for the Linux device manager:

sudo nano /etc/udev/rules.d/51-kinect.rules

Copy and paste:

# ATTR{product}=="Xbox NUI Motor"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02b0", MODE="0666"
# ATTR{product}=="Xbox NUI Audio"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02ad", MODE="0666"
# ATTR{product}=="Xbox NUI Camera"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02ae", MODE="0666"
# ATTR{product}=="Xbox NUI Motor"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02c2", MODE="0666"
# ATTR{product}=="Xbox NUI Motor"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02be", MODE="0666"
# ATTR{product}=="Xbox NUI Motor"
SUBSYSTEM=="usb", ATTR{idVendor}=="045e", ATTR{idProduct}=="02bf", MODE="0666"

OpenCV libraries

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.

It has C++, C, Python, Java and MATLAB interfaces and supports Windows, Linux, Android and Mac OS. OpenCV leans mostly towards real-time vision applications and takes advantage of MMX and SSE instructions when available

Installing OpenCV libraries

OpenCV package is named libopencv. Detailed information about installation is here

• Ubuntu manual installation
  • Getting Latest Stable OpenCV version
  • Go to the page on Sourceforge
  • Download the source tarball and unpack it.
  • Getting the Cutting-edge OpenCV from the Git Repository
  • Launch Git client and clone OpenCV repository
  • In Linux it can be achieved with the following command in Terminal:
    cd ~/<my_working _directory>
git clone https://github.com/Itseez/opencv.git
  • Building OpenCV from Source
  • Create a temporary directory, which we denote as <cmake_binary_dir>, where you want to put the generated Makefiles, project files as well the object files and output binaries.
  • Enter the <cmake_binary_dir> and type
    cmake [<some optional parameters>] <path to the OpenCV source directory>
    For example
    cd ~/opencv
mkdir release
cd release
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
  • Enter the created temporary directory (<cmake_binary_dir>) and proceed with:
    make
sudo make install

Packages dependencies

All drivers and libraries (OpenNI, libfrenect and libopencv) requires some pre-installed software to work properly. Based on Linux systems, the packages dependencies needed are listed as follows.

For OpenNI + NiTE

Run in terminal

$ sudo apt-get install freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev doxygen graphviz mono-complete

For libfreenect

To build libfreenect and run its examples, the following packages are needed:

• libusb >= 1.0.18
• CMake >= 2.6
• GCC > 4.x
• python == 2.*
• OpenGL
• glut
• pthreads

To install those ones run in terminal:

$ sudo apt-get install libusb-1.0-0-dev cmake build-essential git-core libglut3-dev pkg-config libxmu-dev libxi-dev libpthread-stubs0-dev

If you are getting an error saying apt-get cannot find libglut3, you might be on a newer version of a Linux distribution that has freeglut3-* instead of libglut3-*, so your initial apt-get install would look like:

$ sudo apt-get install git-core cmake freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev

For OpenCV

To build OpenCV, the following packages are needed:

• GCC 4.4.X or higher
• CMake 2.6 or higher
• git
• GTK 2.x or higher
• pkg-config
• python 2.6 or higher and Numpy 1.5 or higher
• ffmpeg
• libav
• libdc1394 2x [optional]
• libjpeg-dev, libpng-dev, libtiff-dev, libjasper-dev [optional]

To install those ones run in terminal:

$ sudo apt-get install build-essential cmake git libgtk2.0-dev pkg-config python-dev, python-numpy  libavcodec-dev, libavformat-dev libdc1394-2.x libjpeg-dev libpng-dev libtiff-dev libjasper-dev

libdc1394-2x may vary from 21 to 22, making reference to version 2.1 and 2.2 respectively

Flags needed for development in C++

C and C++ compilers requires certain flags to invoke libraries needed such as libfreenect and libopencv.

Depending on programming enviroment used, flags can de added in different ways. For Eclipse IDE, for instance, adding flags it is:

1. Go to Project > Properties > C/C++ Build > Settings > GCC C++ Linker > Libraries
2. In "libraries (-l)" click on the "Add.." button and enter the librarie name wihtoul "-l" i.e. write freenect instead -lfreenect
3. In "Libraries search path (-L)" must appear the paths where the libraries are referenced, normally
   /usr/lib
   /usr/local/lib

For gcc or g++ in terminal, compile code following this format:

$ g++ <name_of_code.cpp> -o <name_of_excutable_file> `pkg-config --cflags --libs opencv` <flag1> <flag2> ...

The libraries needed are:

• -lm
• -lglut
• -lGL
• -lGLU
• -lpthread
• -lfreenect

If OpenCV flags are not detected using `pkg-config --cflags --libs opencv`. Add these flags:

• -lopencv_calib3d
• -lopencv_contrib
• -lopencv_core
• -lopencv_features2d
• -lopencv_flann
• -lopencv_gpu
• -lopencv_highgui
• -lopencv_imgproc
• -lopencv_legacy
• -lopencv_objdetect
• -lopencv_ocl
• -lopencv_photo
• -lopencv_stitching
• -lopencv_superres
• -lopencv_ts
• -lopencv_video
• -lopencv_videostab
• -lopencv_ml

Using openFrameworks as toolkit for development in C++

openFrameworks is an open source C++ toolkit designed to assist the creative process by providing a simple and intuitive framework for experimentation.

openFrameworks is designed to work as a general purpose glue, and wraps together several commonly used libraries, including:

• ----
  • OpenGL, GLEW, GLUT, libtess2 and cairo for graphics
  • rtAudio, PortAudio, OpenAL and Kiss FFT or FMOD for audio input, output and analysis
  • FreeType for fonts
  • FreeImage for image saving and loading
  • Quicktime, GStreamer and videoInput for video playback and grabbing
  • Poco for a variety of utilities
  • OpenCV for computer vision
  • Assimp for 3D model loading

The code is written to be massively cross-compatible. Right now we support five operating systems (Windows, OSX, Linux, iOS, Android) and four IDEs (XCode, Code::Blocks, and Visual Studio and Eclipse). The API is designed to be minimal and easy to grasp.

Downloading openFrameworks

openFrameworks has both 32 and 64 bit version to work in Linux, Windows, iOS and Android. Downloads are availbale here and consists in a compressed file (tar.gz or .zip) that can be downloaded and unpacked in any directory whit administrator permissions.

For Linux systems, open a terminal and go to the directory where of_v0.X.X_linux_release.tar.gz file was downloaded and enter:

tar -zxvf of_v0.X.X_linux_release.tar.gz

Where v0.X.X has to be modified to the current version of openFrameworks available

Prerequisites for communication wih Kinect

To allow communication with Kinect device, openFrameworks requires libfreenect drivers. Refer to install openKinect Drivers to go through the installation of such drivers

Generating a new openFrameworks project

This space is still pending

Downloading and running project

As project is still in an early stage of development, no public versions are avaiable at the moment. For now, only hpclab server users can clone the project from git.

Downloading project from git repository

Since openFrameworks is more a toolkit rather than a installed software, all its dependencies and libraries needed for development are stored inside the openFramworks folder. In particular, every project created using openFramworks must be created using the Project Generator and saved in the openFramworks_folder->apps->myApps directory.

Because current version of project software was developed using openFrameworks, is necessary to clone the folder from git repository into "myApps" folder.

To create the project folder, in Linux, go into the openFrameworks current directory, open a terminal and enter:

cd apps/Myapps
git clone ssh://username@172.20.150.35/var/repo/Skeleton_tracks_of.git Skeleton_tracks_of

If the project folder is already created and want to update to the latest version, go to the openFrameworks current directory, open a terminal an enter:

cd apps/Myapps/Skeleton_tracks_of
git pull ssh://username@172.20.150.35/var/repo/Skeleton_tracks_of.git

Compiling

The code for the project was written in C++ using openFrameworks toolkit and is located in the Skeleton_tracks_of/src/ directory corresponding to the following files:

• main.cpp
• ofApp.cpp
• ofApp.h

To compile project go to openFrameworks directory, open a terminal and enter:

cd apps/myApps/Skeleton_tracks_of/
make

Executing

To run the application, step in the same directory where project was compiled with make command and enter:

cd bin
./Skeleton_tracks_of

or

make run

---

<- Back