Skip to content
Commits on Source (204)
Expand all Hide whitespace changes
Inline Side-by-side
.gitignore
View file @ a394b22a
CMakeFiles/
CMakeCache.txt
CMakeLists.txt.user
cmake_install.cmake
Makefile
/doc/*.aux
/doc/*.bbl
/doc/*.blg
/doc/*.idx
/doc/*.ilg
/doc/*.ind
/doc/*.log
/doc/*.out
/doc/*.pdf
/doc/*.synctex.gz
/doc/*.toc
/CMakeFiles/
/CMakeCache.txt
/CMakeLists.txt.user
/cmake_install.cmake
/Makefile
*~
build/
/build*
.gitlab-ci.yml
View file @ a394b22a
stages:
- makedoc
- deploy
- deploydoc
- build
job2:
job1:
stage: makedoc
script: mkdir build; cd build; cmake -DCMAKE_BUILD_TYPE=Release ../; make everything; cp doc/libRaptorQ.pdf ~/
script: mkdir build; cd build; cmake -DCMAKE_BUILD_TYPE=Release ../; make docs; cp doc/libRaptorQ-RFC6330.pdf ~/; cp doc/libRaptorQ-RaptorQ.pdf ~/
job2:
stage: deploydoc
script: git diff-tree --no-commit-id --name-only -r HEAD | grep ^doc/ || exit 0; cd /var/lib/gitlab-runner/libRaptorQ.wiki; git pull; cp ~/libRaptorQ-RFC6330.pdf .; cp ~/libRaptorQ-RaptorQ.pdf .; git add libRaptorQ-RFC6330.pdf; git add libRaptorQ-RaptorQ.pdf; git commit -m '(gitlab-runner) updated pdf'; git push;
job3:
stage: deploy
script: git diff-tree --no-commit-id --name-only -r HEAD | grep ^doc/ || exit 0; cd /var/lib/gitlab-runner/libRaptorQ.wiki; git pull; cp ~/libRaptorQ.pdf .; git add libRaptorQ.pdf; git commit -m '(gitlab-runner) updated pdf'; git push;
stage: build
script: mkdir build; cd build; cmake -DCMAKE_BUILD_TYPE=Release ../; make everything;
.gitmodules 0 → 100644
View file @ a394b22a
[submodule "external/lz4"]
path = external/lz4
url = https://github.com/Cyan4973/lz4.git
CHANGELOG
View file @ a394b22a
v1.0.0 (beta):
* C++98 compatibility (untested)
* we are now a header-only library that can be compiled and linked to C/C++98, with RFC and RAW API
* RAW API block size is now typesafe.
* RAW API does not use the interleaving code.
* API reworked to easy manage old/new API in the future
* new, simpler RAW API (single-block)
* computations can be stopped
* precomputation caching
* cached result can be compressed (raw/LZ4)
* RFC API: global multithreading instead of per-class
* RFC compliance
* CLI tool for raw API
* OSX/BSD support
v0.1.10 (unreleased):
* backported MAC/FreeBSD support (and incomplete windows)
* backported RFC compliance
v0.1.9:
* remember to bump up version in the cmake -.-'
* squashed warnings for gcc & clang in 32 bit mode.
v0.1.8:
* someone forgot to update the changelog :p
* was bugfixing only, no new feature
* fixed initializing encoder/decoder with more than 256 blocks
* fixed untriggerable bug on writing decoding data.
v0.1.4:
* finally reached a usable state
CMakeLists.txt
View file @ a394b22a
This diff is collapsed.
CONTRIBUTING.md
View file @ a394b22a
......@@ -4,6 +4,9 @@
Third-party patches are always welcome, for any purpose.
Although the repository is available from GitHub, please
use the main site for bug reporting/merges/discussions.
Please be aware that libRaptorQ is a dual-licensed software,
and thus it's required that all contributors sign a CLA agreement.
The agreement lets us relicense the code and protect us from patents,
......@@ -63,7 +66,7 @@ developer accepted the CLA and is the author of the commit.
* Always sign your commits with the "-s" option, to add the "Signed-off-by" line in the commit message
* Only the main developers are required to GPG-sign the commits.
* Run the tests to make sure you have not broken something by mistake
* Please keep commits short and to the point. Nobody wants to review 2000+ lines of code that touch everything.
## Documentation
......
README deleted 100644 → 0
View file @ e5537ce0
RaptorQ is a Forward Error Correction algorithm designed to deliver your data
efficiently and without retransmissions for lost packets.
After sending K packets of your data as-is, RaptorQ generates as many repair
symbols as you need. Once the receiver has at least K symbols, be it the source
symbols, repair symbols or any combination of the two, it can reconstruct the
whole input it was meant to receive.
This is called a Fountain code, and RaptorQ is the latest and most efficient
code in this category.
libRaptorQ implements RFC6330, which specifies the RaptorQ algorithm.
NOTE: heavy cluster testing is needed to claim compliance with the
RFC. As we do not have the resources, we do not make such a claim.
libRaptorQ is entirely written in C++11, and uses eigen to handle matrix
manipulation.
Currently it's only been tested under Linux, but should work well under
*BSD and MacOSX, too.
Windows *should* work too, provided you use a complier that understands C++11
(hint: you might have some problems with VisualC++, but that's not a C++
compiler anyway)
##############
#
# Developers
#
##############
See the "CONTRIBUTING.md" file.
##############
#
# The Source Code
#
##############
Although things seems to work, no stable release has been released yet.
This means you can only check this out with git.
to check out the repository:
$ git clone https://github.com/LucaFulchir/libRaptorQ.git
you can also get it from our main server:
$ git clone https://www.fenrirproject.org/Luker/libRaptorQ.git
-- GPG source check --
once you have cloned it, it's always a good thing to check the repository gpg
signatures, so you can import my key with:
$ gpg --keyserver pgp.mit.edu --recv-key D42DDF0A
please check the full fingerprint, it should be like this:
$ gpg2 --fingerprint D42DDF0A
pub rsa3072/D42DDF0A 2015-01-01 [expires: 2016-01-01]
Key fingerprint = AB35 E45F 5CA5 E35B 8B55 818F 0157 D133 D42D DF0A
uid [ unknown] Luca Fulchir (2015 key) <luker@fenrirproject.org>
Now you have the source, and the key, it's enough to check the signature of the
last commit:
$ git log -n 1 --show-signature
The important part is that you get something like this:
gpg: Signature made Fri 27 Mar 2015 20:59:59 CET using RSA key ID D42DDF0A
gpg: Good signature from "Luca Fulchir (2015 key) <luker@fenrirproject.org>"
[unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: AB35 E45F 5CA5 E35B 8B55 818F 0157 D133 D42D DF0A
Author: Luca Fulchir <luker@fenrirproject.org>
And as long as you got the right key, and you find the "gpg: Goog signature",
you can be sure you have the right code.
TDB: The repository is using the maintainer private key for now,
it will start using a dedicated key in the (near) future.
##############
#
# Install
#
##############
Before building libRaptorQ, make sure your system has the eigen3 library
headers, or it won't compile.
Eigen3 is the only dependency of libRaptorQ
The build system uses CMake. So enter the source directory and we'll create a
directory and build everything there:
$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ../
$ make -j 4
Optional targets are available:
$ make tests examples docs
where:
tests: benchmarks, rfc tests.
examples: C/C++ examples
docs: LATEX documentation.
or simply:
$ make everything
finally, install everything
$ make install
libRaptorQ uses deterministic (reproducible) builds,
so if you compile it twice, or on two different computers
(but with the same compiler), the hash of the resulting
libraries will be the same.
The only configuration that breaks deterministic builds
currently is the clang + profiling one. It probably can not
be fixed.
You can customize the CMake build with tthe following variables:
PROFILING yes/no: Default:yes. Activate or deactivate profiling.
Profiling compiles everything, then runs a test to see
which code paths are more used. Then it recompiles everything
but optimizing for for those code paths.
Only for gcc/clang.
LTO yes/no: Default:yes. Activate or deactivate Link time Optimization
Makes the library smaller and better optimized.
Only for gcc/clang.
STDLIB libc++,libstdc++. Choose the standard library to use.
Note: gcc can only use its own libstdc++
CMAKE_C_COMPILER gcc, clang...
CMAKE_CXX_COMPILER choose between g++ or clang++.
CMAKE_BUILD_TYPE Debug,MinSizeRel,Release,RelWithDebInfo
CMAKE_INSTALL_PREFIX Default: /usr/local
##############
#
# Using libRaptorQ
#
##############
Compile the documentation:
$ make docs
or simply visit:
https://www.fenrirproject.org/projects/libraptorq/documents
for the full up-to-date documentation.
includes the information in this readme, plus a quick description
of the RaptorQ algorithm, and the C and C++ interfaces.
README.md 0 → 100644
View file @ a394b22a
# RaptorQ #
[![build status](https://www.fenrirproject.org/Luker/libRaptorQ/badges/master/build.svg)](https://www.fenrirproject.org/Luker/libRaptorQ/commits/master)
[![Donate](https://img.shields.io/badge/Donate-PayPal-blue.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=79L8P7TXWFQK4)
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg)](https://patreon.com/Luker)
[Homepage](https://www.fenrirproject.org/Luker/libRaptorQ/wikis/home)
old release: **v0.1.10**
> Note: v0.1.10 is not compatible with prefivous versions, which were not RFC compliant!
current release: **v1.0.0-rc2** (use this in new projects)
RaptorQ is a **Forward Error Correction** algorithm designed to deliver your data
efficiently and without retransmissions for lost packets.
After sending K packets of your data as-is, RaptorQ generates as many repair
symbols as you need. Once the receiver has at least K symbols, be it the source
symbols, repair symbols or any combination of the two, it can reconstruct the
whole input it was meant to receive.
This is called a Fountain code, and RaptorQ is the latest and most efficient
code in this category.
**libRaptorQ** implements **RFC6330**, which specifies the RaptorQ algorithm.
Since the RFC is really complex, there is also a simpler (and maybe slightly faster)
RAW API that you can use. The RFC API is not recommended due to the RFC complexity.
libRaptorQ is a header-only library written in *C++11*, and uses eigen to handle matrix
manipulation.
Although header-only, the library can be compiled to create shared and static libraries
with both C and C++98 compatibility.
Currently it's only been tested under Linux, but should work well under
*BSD and MacOSX, too.
Unfortunately Windows is not supported. support for <future> is broken,
so you can not compile it with Visual Studio 2015.
## Features
- RFC6330 API (complex, not recommended)
- RAW API (simpler, recommended)
- multi language:
- header only C++11
- compiled, C
- compiled, C++98
- cached precomputations (configurable)
## Developers ##
See the [CONTRIBUTING](CONTRIBUTING.md) file.
## The Source Code ##
Although things seems to work, no stable release has been released yet.
This means you can only check this out with git.
to check out the repository:
``$ git clone https://github.com/LucaFulchir/libRaptorQ.git``
you can also get it from our main server:
``$ git clone https://www.fenrirproject.org/Luker/libRaptorQ.git``
### GPG source check ###
Once you have cloned it, it's always a good thing to check the repository gpg
signatures, so you can import my key with:
new long term key: **7393 DAD2 55BE B575 1DBD A04A B11C D823 BA27 8C85**
``$ gpg --keyserver pgp.mit.edu --recv-key 7393DAD255BEB5751DBDA04AB11CD823BA278C85``
2016 key:
``$ gpg --keyserver pgp.mit.edu --recv-key F61F6137``
2015 key:
``$ gpg --keyserver pgp.mit.edu --recv-key D42DDF0A``
please check the full fingerprint.
Now you have the source, and the key, it's enough to check the signature of the
last commit:
``$ git log -n 1 --show-signature``
As long as you got the right key, and you find the "gpg: Goog signature",
you can be sure you have the right code.
## Install ##
#### Dependencies ####
libRaptorQ depends from **Eigen3** and **LZ4**
In case your system does not have Eigen3, Eigenv3.2.8 is included
in the source files, so you do not need it.
LZ4 is included as a git submodule, so if you do not have it, run:
```git submodule init```
```git submodule update```
To get the library, which will be statically linked and **NOT INSTALLED**
on your system.
#### Building ####
The build system uses CMake. So enter the source directory and we'll create a
directory and build everything there:
```bash
$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ../
$ make -j 4
```
Optional targets are available:
``$ make tests examples docs``
where:
* tests: benchmarks, rfc tests.
* examples: C/C++ examples
* docs: LATEX documentation.
or simply:
``$ make -j 4 everything``
finally, install everything
``$ make install``
libRaptorQ uses **deterministic (reproducible) builds**,
so if you compile it twice, or on two different computers
(but with the same compiler), the hash of the resulting
libraries will be the same.
_There are combinations of compiler and LTO/Profiling that
break deterministic builds, so check the cmake warnings._
You can customize the CMake build with the following variables:
```
PROFILING ON/OFF: Default:ON. Activate or deactivate profiling.
Profiling compiles everything, then runs a test to see
which code paths are more used. Then it recompiles everything
but optimizing for for those code paths.
Only for gcc/clang.
LTO ON/OFF: Default:ON. Activate or deactivate Link time Optimization
Makes the library smaller and better optimized.
Only for gcc/clang.
CLANG_STDLIB ON/OFF: Default:OFF. use clang's libc++
Note: only clang can use its standard library
USE_LZ4 ON/OFF: use the lz4 compression for caching precomputations.
Default: ON
CLI ON/OFF Build Command Line Interface tools.
CMAKE_C_COMPILER gcc, clang...
CMAKE_CXX_COMPILER choose between g++ or clang++.
RQ_LINKER gold/ld/bsd Choose your linker. Default:autodetect.
CMAKE_BUILD_TYPE Debug,MinSizeRel,Release,RelWithDebInfo
CMAKE_INSTALL_PREFIX Default: /usr/local
```
## Using libRaptorQ ##
For the C++11, header-only version, you can include:
- "RaptorQ/RaptorQ_v1_hdr.hpp"
- "RaptorQ/RFC6330_v1_hdr.hpp"
For the linked, C+98/C++11 API:
- "RaptorQ/RaptorQ_v1.hpp"
- "RaptorQ/RFC6330_v1.hpp"
For the linked, C API:
- "RaptorQ/RaptorQ.h"
- "RaptorQ/RFC6330.h"
The C++ api is completely in sync between the linked and header-only version,
so you can switch between the two just by changing the included header file.
You can compile a PDF of the documentation by doing:
``$ make docs``
Or you can simply visit the [wiki](https://www.fenrirproject.org/Luker/libRaptorQ/wikis/libRaptorQ.pdf)
for the full up-to-date documentation.
cmake/Findstd_clang.cmake → cmake/FindRQ_LZ4.cmake
View file @ a394b22a
......@@ -18,33 +18,55 @@
# along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
#
FIND_PATH(CLANG_STD_INCLUDE_DIR
NAMES unordered_map
PATH_SUFFIXES include/c++ include/c++/v1
PATHS "${PROJECT_SOURCE_DIR}/../libc++/"
${CLANG_STD_ROOT}
$ENV{CLANG_STD_ROOT}
/usr/local/
/usr/
FIND_PATH(RQ_LZ4_INCLUDE_DIR
NAMES lz4.h
PATH_SUFFIXES include/ include/lz4/ lz4/
PATHS
${LZ4_ROOT}
$ENV{LZ4_ROOT}
/usr/
/usr/local/
${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib
)
message("Found clang stdandard library at: ${CLANG_STD_INCLUDE_DIR}.")
FIND_LIBRARY(RQ_LZ4_LIB
NAMES lz4
PATH_SUFFIXES lib/
PATHS
${LZ4_ROOT}
$ENV{LZ4_ROOT}
/usr/
/usr/local/
)
IF(CLANG_STD_INCLUDE_DIR)
SET(CLANG_STD_FOUND TRUE)
ELSE(CLANG_STD_INCLUDE_DIR)
SET(CLANG_STD_FOUND FALSE)
ENDIF(CLANG_STD_INCLUDE_DIR)
IF(RQ_LZ4_INCLUDE_DIR)
SET(RQ_LZ4_FOUND TRUE)
ELSE(RQ_LZ4_INCLUDE_DIR)
SET(RQ_LZ4_FOUND FALSE)
ENDIF(RQ_LZ4_INCLUDE_DIR)
IF(CLANG_STD_FOUND)
MESSAGE(STATUS "Found clang standard library in ${CLANG_STD_INCLUDE_DIR}")
ELSE(CLANG_STD_FOUND)
IF(CLANG_STD_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find the clang standard library")
ENDIF(CLANG_STD_FIND_REQUIRED)
ENDIF(CLANG_STD_FOUND)
IF(RQ_LZ4_FOUND)
IF(RQ_LZ4_USE_OWN)
# force our own lz4 libary
SET(RQ_LZ4_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib)
ENDIF()
MESSAGE(STATUS "Found lz4 in ${RQ_LZ4_INCLUDE_DIR}")
# we need both headers and library, or we use our own
IF(RQ_LZ4_INCLUDE_DIR MATCHES "${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib" OR RQ_LZ4_LIB MATCHES "RQ_LZ4_LIB-NOTFOUND")
MESSAGE(WARNING "We will build our own lz4 library and statically link it.")
SET(RQ_BUILD_LZ4 TRUE)
ENDIF()
ELSE(RQ_LZ4_FOUND)
IF(RQ_LZ4_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"lz4\" library."
" Please install the \"lz4\" library"
" or at least run:\n"
" cd ${CMAKE_CURRENT_SOURCE_DIR}\n"
" git submodule init\n"
" git submodule update\n")
ENDIF(RQ_LZ4_FIND_REQUIRED)
ENDIF(RQ_LZ4_FOUND)
MARK_AS_ADVANCED(
CLANG_STD_INCLUDE_DIR
RQ_LZ4_INCLUDE_DIR
)
cmake/Findeigen.cmake
View file @ a394b22a
......@@ -19,31 +19,31 @@
#
FIND_PATH(EIGEN3_INCLUDE_DIR
NAMES Eigen
PATH_SUFFIXES include/ include/eigen3/ eigen3/
PATHS "${PROJECT_SOURCE_DIR}"
${EIGEN3_ROOT}
$ENV{EIGEN3_ROOT}
/usr/
NAMES Eigen
PATH_SUFFIXES include/ include/eigen3/ eigen3/
PATHS
${EIGEN3_ROOT}
$ENV{EIGEN3_ROOT}
/usr/
/usr/local/
${PROJECT_SOURCE_DIR}/external/eigen3
)
message("Found eigen3 at: ${EIGEN3_INCLUDE_DIR}.")
IF(EIGEN3_INCLUDE_DIR)
SET(EIGEN3_FOUND TRUE)
SET(EIGEN3_FOUND TRUE)
ELSE(EIGEN3_INCLUDE_DIR)
SET(EIGEN3_FOUND FALSE)
SET(EIGEN3_FOUND FALSE)
ENDIF(EIGEN3_INCLUDE_DIR)
IF(EIGEN3_FOUND)
MESSAGE(STATUS "Found eigen3 in ${EIGEN3_INCLUDE_DIR}")
MESSAGE(STATUS "Found eigen3 in ${EIGEN3_INCLUDE_DIR}")
ELSE(EIGEN3_FOUND)
IF(EIGEN3_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"eigen3\" library")
ENDIF(EIGEN3_FIND_REQUIRED)
IF(EIGEN3_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"eigen3\" library")
ENDIF(EIGEN3_FIND_REQUIRED)
ENDIF(EIGEN3_FOUND)
MARK_AS_ADVANCED(
EIGEN3_INCLUDE_DIR
EIGEN3_INCLUDE_DIR
)
cmake/Findgit.cmake
View file @ a394b22a
......@@ -18,29 +18,29 @@
# along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
#
FIND_PROGRAM(GIT_EXECUTABLE git
PATHS
${PATH}
"C:/Program Files/Git/bin"
"C:/Program Files (x86)/Git/bin"
DOC "git command line client"
PATHS
${PATH}
"C:/Program Files/Git/bin"
"C:/Program Files (x86)/Git/bin"
DOC "git command line client"
)
IF(GIT_EXECUTABLE)
SET(GIT_FOUND TRUE)
SET(GIT_FOUND TRUE)
ELSE(GIT_EXECUTABLE)
SET(GIT_FOUND FALSE)
SET(GIT_FOUND FALSE)
ENDIF(GIT_EXECUTABLE)
IF(GIT_FOUND)
MESSAGE(STATUS "Found git in ${GIT_EXECUTABLE}")
MESSAGE(STATUS "Found git in ${GIT_EXECUTABLE}")
ELSE(GIT_FOUND)
IF(GIT_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"git\" command")
ENDIF(GIT_REQUIRED)
IF(GIT_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"git\" command")
ENDIF(GIT_REQUIRED)
ENDIF(GIT_FOUND)
MARK_AS_ADVANCED(
GIT_EXECUTABLE
GIT_EXECUTABLE
)
......
cmake/Findstd_gcc.cmake deleted 100644 → 0
View file @ e5537ce0
#
# Copyright (c) 2015, Luca Fulchir<luca@fulchir.it>, All rights reserved.
#
# This file is part of libRaptorQ.
#
# libRaptorQ is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation, either version 3
# of the License, or (at your option) any later version.
#
# libRaptorQ is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# and a copy of the GNU Lesser General Public License
# along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
#
file(GLOB sub-dir /usr/include/c++/[0-9].[0-9]*.[0-9]*)
foreach(dir ${sub-dir})
if(IS_DIRECTORY ${dir})
set(GCC_PATH_SUFFIXES ${GCC_PATH_SUFFIXES};${dir})
endif()
endforeach()
FIND_PATH(GCC_STD_INCLUDE_DIR
NAMES unordered_map
PATH_SUFFIXES / ${GCC_PATH_SUFFIXES}
PATHS "${PROJECT_SOURCE_DIR}/../stdlib/"
${GCC_STD_ROOT}
$ENV{GCC_STD_ROOT}
/
)
message("Found gcc stdandard library at: ${GCC_STD_INCLUDE_DIR}.")
IF(GCC_STD_INCLUDE_DIR)
SET(GCC_STD_FOUND TRUE)
ELSE(GCC_STD_INCLUDE_DIR)
SET(GCC_STD_FOUND FALSE)
ENDIF(GCC_STD_INCLUDE_DIR)
IF(GCC_STD_FOUND)
MESSAGE(STATUS "Found gcc standard library in ${GCC_STD_INCLUDE_DIR}")
ELSE(GCC_STD_FOUND)
IF(GCC_STD_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find the gcc standard library")
ENDIF(GCC_STD_FIND_REQUIRED)
ENDIF(GCC_STD_FOUND)
MARK_AS_ADVANCED(
GCC_STD_INCLUDE_DIR
)
cmake/compiler_flags.cmake 0 → 100644
View file @ a394b22a
#
# Copyright (c) 2015-2016, Luca Fulchir<luca@fulchir.it>, All rights reserved.
#
# This file is part of "libRaptorQ".
#
# libRaptorQ is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation, either version 3
# of the License, or (at your option) any later version.
#
# libRaptorQ is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# and a copy of the GNU Lesser General Public License
# along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
#
include(CheckCCompilerFlag)
include(CheckCXXCompilerFlag)
#string of multiple flags to list
string(REPLACE " " ";" RQ_C_FLAGS "${CMAKE_C_FLAGS}")
string(REPLACE " " ";" RQ_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
##############################################
#### check for UBSAN sanitizing support ######
##############################################
if(CMAKE_BUILD_TYPE MATCHES "Debug")
check_c_compiler_flag("-fsanitize=undefined" RQ_FLAG_C_UBSAN)
check_cxx_compiler_flag("-fsanitize=undefined" RQ_FLAG_CXX_UBSAN)
if (RQ_FLAG_C_UBSAN AND RQ_FLAG_CXX_UBSAN)
set(RQ_ENABLE_UBSAN TRUE)
message(STATUS "UBSAN sanitizing support enabled")
else()
message(STATUS "UBSAN sanitizing support disabled")
endif()
add_definitions(-DRQ_DEBUG=true)
else()
add_definitions(-DRQ_DEBUG=false)
endif()
###################
## Compiler options
###################
#gnu options
set(RQ_GNU_C_OPTIONS ${RQ_DETERMINISTIC} -std=c11
-Wno-unknown-pragmas -Wall -Wextra -pedantic -Wno-padded
-fstack-protector-all -fstrict-aliasing -fwrapv -fvisibility=hidden
-Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict
-Wnull-dereference -Wdouble-promotion -Wshadow -Wformat=2)
set(RQ_GNU_CXX_OPTIONS ${RQ_DETERMINISTIC} -std=c++11
-fno-rtti -fno-exceptions -Wno-unknown-pragmas -Wall -Wextra
-pedantic -Wno-padded -Wno-unknown-pragmas -fstack-protector-all
-fstrict-aliasing -fwrapv -fvisibility=hidden -fvisibility-inlines-hidden
-Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict
-Wnull-dereference -Wuseless-cast -Wdouble-promotion -Wshadow -Wformat=2
-Wno-duplicated-branches)
# GCC internal compiler errors with:
# -fsanitize=undefined
# -fsanitize=null
# -fsanitize=signed-integer-overflow
set(RQ_GNU_C_DEBUG -O0 -g -Wno-aggressive-loop-optimizations -ftrapv )
set(RQ_GNU_CXX_DEBUG -O0 -g -Wno-aggressive-loop-optimizations -ftrapv )
set(RQ_GNU_C_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=vla-bound -fsanitize=return -fisolate-erroneous-paths-dereference -fisolate-erroneous-paths-attribute )
set(RQ_GNU_CXX_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=vla-bound -fsanitize=return -fisolate-erroneous-paths-dereference -fisolate-erroneous-paths-attribute )
set(RQ_GNU_C_MINSIZEREL -Os )
set(RQ_GNU_CXX_MINSIZEREL -Os )
set(RQ_GNU_C_RELEASE -Ofast -DNDEBUG -fwrapv -ftree-loop-distribution -funroll-loops )
set(RQ_GNU_CXX_RELEASE -Ofast -DNDEBUG -fwrapv -ftree-loop-distribution -funroll-loops )
set(RQ_GNU_C_RELWITHDEBINFO -g -Ofast -fwrapv -ftree-loop-distribution -funroll-loops )
set(RQ_GNU_CXX_RELWITHDEBINFO -g -Ofast -fwrapv -ftree-loop-distribution -funroll-loops )
# clang options
set(RQ_CLANG_C_OPTIONS ${RQ_DETERMINISTIC} -std=c11
-fno-math-errno -Wall -Wextra -pedantic -Weverything -Wno-padded
-fstack-protector-all -fstrict-aliasing -Wformat -Wformat-security
-Wno-disabled-macro-expansion -fvisibility=hidden -fvisibility-inlines-hidden
-Wdouble-promotion -Wshadow -Wformat=2 -Wnull-dereference)
set(RQ_CLANG_CXX_OPTIONS ${RQ_STDLIB_FLAG} ${RQ_DETERMINISTIC}
-std=c++11 -fno-rtti -fno-exceptions -fno-math-errno
-Wall -pedantic -Weverything -Wno-c++98-compat-pedantic -Wno-c++98-compat
-Wno-padded -Wno-unknown-pragmas -fstack-protector-all -fstrict-aliasing
-Wformat -Wformat-security -fvisibility=hidden -fvisibility-inlines-hidden
-Wno-documentation -Wno-documentation-unknown-command
-Wdouble-promotion -Wshadow -Wformat=2 -Wnull-dereference)
set(RQ_CLANG_C_DEBUG -O0 -g )
set(RQ_CLANG_CXX_DEBUG -O0 -g )
set(RQ_CLANG_C_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=vla-bound -fsanitize=return )
set(RQ_CLANG_CXX_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=vla-bound -fsanitize=return )
set(RQ_CLANG_C_MINSIZEREL -Os )
set(RQ_CLANG_CXX_MINSIZEREL -Os )
set(RQ_CLANG_C_RELEASE -Ofast -DNDEBUG -fwrapv )
set(RQ_CLANG_CXX_RELEASE -Ofast -DNDEBUG -fwrapv )
set(RQ_CLANG_C_RELWITHDEBINFO -g -Ofast -fwrapv )
set(RQ_CLANG_CXX_RELWITHDEBINFO -g -Ofast -fwrapv )
# msvc flags. todo?
set(RQ_MSVC_C_OPTIONS /wd4068)
set(RQ_MSVC_CXX_OPTIONS /wd4068)
###################
# combine the above
###################
# note: generator expansions do not work on "set(..)"
if(CMAKE_CXX_COMPILER_ID MATCHES "Clang")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_CXX_FLAGS} ${RQ_CLANG_CXX_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_DEBUG})
if (RQ_ENABLE_UBSAN)
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_DEBUG_SANITIZE})
set(RQ_UBSAN ${RQ_CLANG_CXX_DEBUG_SANITIZE})
endif()
elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_MINSIZEREL})
elseif(CMAKE_BUILD_TYPE MATCHES "Release")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_RELEASE})
elseif(CMAKE_BUILD_TYPE MATCHES "RelWithDebInfo")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_RELWITHDEBINFO})
endif()
elseif(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_CXX_FLAGS} ${RQ_GNU_CXX_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_DEBUG})
if (RQ_ENABLE_UBSAN)
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_DEBUG_SANITIZE})
set(RQ_UBSAN ${RQ_GNU_CXX_DEBUG_SANITIZE})
endif()
elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_MINSIZEREL})
elseif(CMAKE_BUILD_TYPE MATCHES "Release")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_RELEASE})
elseif(CMAKE_BUILD_TYPE MATCHES "RelWithDebInfo")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_RELWITHDEBINFO})
endif()
elseif(CMAKE_CXX_COMPILER_ID MATCHES "MSVC")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_CXX_FLAGS} ${RQ_MSVC_CXX_OPTIONS})
endif()
if(CMAKE_C_COMPILER_ID MATCHES "Clang")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_C_FLAGS} ${RQ_CLANG_C_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_DEBUG})
if (RQ_ENABLE_UBSAN)
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_DEBUG_SANITIZE})
endif()
elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_MINSIZEREL})
elseif(CMAKE_BUILD_TYPE MATCHES "Release")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_RELEASE})
elseif(CMAKE_BUILD_TYPE MATCHES "RelWithDebInfo")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_RELWITHDEBINFO})
endif()
elseif(CMAKE_C_COMPILER_ID MATCHES "GNU")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_C_FLAGS} ${RQ_GNU_C_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_DEBUG})
if (RQ_ENABLE_UBSAN)
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_DEBUG_SANITIZE})
endif()
elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_MINSIZEREL})
elseif(CMAKE_BUILD_TYPE MATCHES "Release")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_RELEASE})
elseif(CMAKE_BUILD_TYPE MATCHES "RelWithDebInfo")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_RELWITHDEBINFO})
endif()
elseif(CMAKE_C_COMPILER_ID MATCHES "MSVC")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_C_FLAGS} ${RQ_MSVC_C_OPTIONS})
endif()
########################################
## Test all the previously enabled flags
########################################
set(RQ_C_COMPILER_FLAGS "")
set(RQ_CXX_COMPILER_FLAGS "")
# check each C flag
foreach(flag ${RQ_LIST_C_COMPILER_FLAGS})
string(REPLACE "-" "_" RQ_FLAG2 ${flag})
string(REPLACE "+" "_" RQ_FLAG1 ${RQ_FLAG2})
string(REPLACE "=" "_" RQ_FLAG ${RQ_FLAG1})
check_c_compiler_flag(${flag} RQ_FLAG_C_${RQ_FLAG})
if(RQ_FLAG_C_${RQ_FLAG})
set(C_COMPILER_FLAGS ${C_COMPILER_FLAGS} ${flag})
else()
message(WARNING "flag \"${flag}\" can't be used in your C compiler")
endif()
endforeach()
# check each c++ flag
foreach(flag ${RQ_LIST_CXX_COMPILER_FLAGS})
string(REPLACE "-" "_" RQ_FLAG2 ${flag})
string(REPLACE "+" "_" RQ_FLAG1 ${RQ_FLAG2})
string(REPLACE "=" "_" RQ_FLAG ${RQ_FLAG1})
check_cxx_compiler_flag(${flag} RQ_FLAG_CXX_${RQ_FLAG})
if(RQ_FLAG_CXX_${RQ_FLAG})
set(CXX_COMPILER_FLAGS ${CXX_COMPILER_FLAGS} ${flag})
else()
message(WARNING "flag \"${flag}\" can't be used in your CXX compiler")
endif()
endforeach()
cmake/guess_architecture.cmake
View file @ a394b22a
#
# Copyright (c) 2015, Luca Fulchir<luca@fulchir.it>, All rights reserved.
# Copyright (c) 2015-2018, Luca Fulchir<luca@fulchir.it>, All rights reserved.
#
# This file is part of libRaptorQ.
#
......@@ -57,5 +57,3 @@ ELSE()
ENDIF()
ENDIF()
cppcheck.suppress
View file @ a394b22a
// Performance:
// Octet is a sinble byte. value is faster than reference.
// Octet is a single byte. value is faster than reference.
passedByValue:src/Parameters.hpp:73
passedByValue:src/Parameters.hpp:78
passedByValue:src/Parameters.hpp:83
......@@ -13,6 +13,11 @@ passedByValue:src/Parameters.hpp:132
passedByValue:src/Interleaver.hpp:170
passedByValue:src/Interleaver.hpp:263
passedByValue:src/RaptorQ.hpp:143
passedByValue:src/RaptorQ.hpp:161
passedByValue:src/De_Interleaver.hpp:35
// variable is read by different threads
unreadVariable:test/rfc_test.cpp:569
functionStatic:src/Parameters.hpp:116
noExplicitConstructor:src/Parameters.hpp:69
......@@ -21,5 +26,11 @@ noExplicitConstructor:src/Parameters.hpp:69
// which we do, valgrind never gives us "uninitalized value"
uninitMemberVar:src/Parameters.hpp:68
unusedFunction:src/cRaptorQ.cpp:270
unusedFunction:src/cRaptorQ.cpp:596
unusedFunction:src/cRaptorQ.cpp:452
unusedFunction:src/cRaptorQ.cpp:752
unusedFunction:src/cRaptorQ.cpp:385
missingIncludeSystem
cppcheck_howto
View file @ a394b22a
cppcheck --std=c++11 --enable=all --inconclusive --xml-version=2 --suppressions cppcheck.suppress src/ test/
cppcheck --std=c++11 --enable=all --inconclusive --xml-version=2 --suppressions-list=cppcheck.suppress src/ test/
doc/CMakeLists.txt
View file @ a394b22a
......@@ -23,17 +23,24 @@ CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
INCLUDE(../cmake/UseLATEX.cmake)
IF (LATEX_COMPILER AND PDFLATEX_COMPILER AND MAKEINDEX_COMPILER)
ADD_LATEX_DOCUMENT(libRaptorQ.tex
INPUTS GNU_FDL.tex
USE_INDEX
MANGLE_TARGET_NAMES
NO_DEFAULT
)
ADD_LATEX_DOCUMENT(libRaptorQ-RFC6330.tex
INPUTS GNU_FDL.tex
USE_INDEX
MANGLE_TARGET_NAMES
NO_DEFAULT
)
ADD_LATEX_DOCUMENT(libRaptorQ-RaptorQ.tex
INPUTS GNU_FDL.tex
USE_INDEX
MANGLE_TARGET_NAMES
NO_DEFAULT
)
ADD_CUSTOM_TARGET(docs DEPENDS libRaptorQ_pdf)
ADD_CUSTOM_TARGET(docs DEPENDS libRaptorQ-RFC6330_pdf libRaptorQ-RaptorQ_pdf)
ELSE()
ADD_CUSTOM_TARGET(docs)
ENDIF()
INSTALL(FILES ${CMAKE_BINARY_DIR}/doc/libRaptorQ.pdf DESTINATION share/doc/ OPTIONAL)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/doc/libRaptorQ-RFC6330.pdf DESTINATION share/doc/ OPTIONAL)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/doc/libRaptorQ-RaptorQ.pdf DESTINATION share/doc/ OPTIONAL)
doc/libRaptorQ-RFC6330.tex 0 → 100644
View file @ a394b22a
This diff is collapsed.
examples/example_cpp_raw.cpp 0 → 100644
View file @ a394b22a
/*
* Copyright (c) 2015-2018, Luca Fulchir<luker@fenrirproject.org>,
* All rights reserved.
*
* This file is part of "libRaptorQ".
*
* libRaptorQ is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as
* published by the Free Software Foundation, either version 3
* of the License, or (at your option) any later version.
*
* libRaptorQ is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* and a copy of the GNU Lesser General Public License
* along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
*/
#include "../src/RaptorQ/RaptorQ_v1_hdr.hpp"
#include <algorithm>
#include <fstream>
#include <iostream>
#include <limits>
#include <random>
#include <stdlib.h>
#include <vector>
// Demonstration of how to use the C++ RAW interface
// it's pretty simple, we generate some input,
// then encode, drop some packets (source and repair)
// and finally decode everything.
// rename the main namespace for ease of use
namespace RaptorQ = RaptorQ__v1;
// mysize is bytes.
bool test_rq (const uint32_t mysize, std::mt19937_64 &rnd,
float drop_probability,
const uint8_t overhead);
// the "overhead" variable tells us how many symbols more than the
// minimum we will generate. RaptorQ can not always decode a block,
// but there is a small probability that it will fail.
// More overhead => less probability of failure
// overhead 0 => 1% failures
// overhead 1 => 0.01% failures
// overhead 2 => 0.0001% failures
// etc... as you can see, it make little sense to work with more than 3-4
// overhead symbols, but at least one should be considered
bool test_rq (const uint32_t mysize, std::mt19937_64 &rnd,
float drop_probability,
const uint8_t overhead)
{
// the actual input.
std::vector<uint8_t> input;
input.reserve (mysize);
// initialize vector with random data
// distr should be "uint8_t". But visual studio does not like it, so
// we use uint16_t
// generate a random distribution between all values of uint8_t
std::uniform_int_distribution<int16_t> distr (0,
std::numeric_limits<uint8_t>::max());
// fill our input with random data
for (size_t idx = 0; idx < mysize; ++idx) {
input.push_back (static_cast<uint8_t> (distr(rnd)));
}
// the input will be divided in blocks of 4 bytes.
// it's a bit low, but this is just an example.
// NOTE: the symbol size must be a multiple of the container size.
// since sizeof(uint8_t) == 1 and 4 is a multiple of 1, we are safe.
const uint16_t symbol_size = 4; // bytes
// how many symbols do we need to encode all our input in a single block?
auto min_symbols = (input.size() * sizeof(uint8_t)) / symbol_size;
if ((input.size() * sizeof(uint8_t)) % symbol_size != 0)
++min_symbols;
// convert "symbols" to a typesafe equivalent, RaptorQ::Block_Size
// This is needed becouse not all numbers are valid block sizes, and this
// helps you choose the right block size
RaptorQ::Block_Size block = RaptorQ::Block_Size::Block_10;
for (auto blk : *RaptorQ::blocks) {
// RaptorQ::blocks is a pointer to an array, just scan it to find your
// block.
if (static_cast<uint16_t> (blk) >= min_symbols) {
block = blk;
break;
}
}
// now initialize the encoder.
// the input for the encoder is std::vector<uint8_t>
// the output for the encoder is std::vector<uint8_t>
// yes, you can have different types, but most of the time you will
// want to work with uint8_t
RaptorQ::Encoder<typename std::vector<uint8_t>::iterator,
typename std::vector<uint8_t>::iterator> enc (
block, symbol_size);
// give the input to the encoder. the encoder answers with the size of what
// it can use
if (enc.set_data (input.begin(), input.end()) != mysize) {
std::cout << "Could not give data to the encoder :(\n";
return false;
}
// actual symbols. you could just use static_cast<uint16_t> (blok)
// but this way you can actually query the encoder.
uint16_t _symbols = enc.symbols();
// print some stuff in output
std::cout << "Size: " << mysize << " symbols: " <<
static_cast<uint32_t> (_symbols) <<
" symbol size: " <<
static_cast<int32_t>(enc.symbol_size()) << "\n";
// RQ need to do its magic on the input before you can ask the symbols.
// multiple ways to do this are available.
// The simplest is to run the computation and block everything until
// the work has been done. Not a problem for small sizes (<200),
// but big sizes will take **a lot** of time, consider running this with the
// asynchronous calls
if (!enc.compute_sync()) {
// if this happens it's a bug in the library.
// the **Decoder** can fail, but the **Encoder** can never fail.
std::cout << "Enc-RaptorQ failure! really bad!\n";
return false;
}
// the probability that a symbol will be dropped.
if (drop_probability > static_cast<float> (90.0))
drop_probability = 90.0; // this is still too high probably.
// we will store here all encoded and transmitted symbols
// std::pair<symbol id (esi), symbol data>
using symbol_id = uint32_t; // just a better name
std::vector<std::pair<symbol_id, std::vector<uint8_t>>> received;
{
// in this block we will generate the symbols that will be sent to
// the decoder.
// a block of size X will need at least X symbols to be decoded.
// we will randomly drop some symbols, but we will keep generating
// repari symbols until we have the required number of symbols.
std::uniform_real_distribution<float> drop_rnd (0.0, 100.0);
uint32_t received_tot = 0;
// Now get the source symbols.
// source symbols are specials because they contain the input data
// as-is, so if you get all of these, you don't need repair symbols
// to make sure that we are using the decoder, drop the first
// source symbol.
auto source_sym_it = enc.begin_source();
++source_sym_it; // ignore the first soure symbol (=> drop it)
source_sym_it++;
for (; source_sym_it != enc.end_source(); ++source_sym_it) {
// we save the symbol here:
// make sure the vector has enough space for the symbol:
// fill it with zeros for the size of the symbol
std::vector<uint8_t> source_sym_data (symbol_size, 0);
// save the data of the symbol into our vector
auto it = source_sym_data.begin();
auto written = (*source_sym_it) (it, source_sym_data.end());
if (written != symbol_size) {
// this can only happen if "source_sym_data" did not have
// enough space for a symbol (here: never)
std::cout << written << "-vs-" << symbol_size <<
" Could not get the whole source symbol!\n";
return false;
}
// can we keep this symbol or do we randomly drop it?
float dropped = drop_rnd (rnd);
if (dropped <= drop_probability) {
continue; // start the cycle again
}
// good, the symbol was received.
++received_tot;
// add it to the vector of received symbols
symbol_id tmp_id = (*source_sym_it).id();
received.emplace_back (tmp_id, std::move(source_sym_data));
}
std::cout << "Source Packet lost: " << enc.symbols() - received.size()
<< "\n";
//--------------------------------------------
// we finished working with the source symbols.
// now we need to transmit the repair symbols.
auto repair_sym_it = enc.begin_repair();
auto max_repair = enc.max_repair(); // RaptorQ can theoretically handle
// infinite repair symbols
// but computers are not so infinite
// we need to have at least enc.symbols() + overhead symbols.
for (; received.size() < (enc.symbols() + overhead) &&
repair_sym_it != enc.end_repair (max_repair);
++repair_sym_it) {
// we save the symbol here:
// make sure the vector has enough space for the symbol:
// fill it with zeros for the size of the symbol
std::vector<uint8_t> repair_sym_data (symbol_size, 0);
// save the data of the symbol into our vector
auto it = repair_sym_data.begin();
auto written = (*repair_sym_it) (it, repair_sym_data.end());
if (written != symbol_size) {
// this can only happen if "repair_sym_data" did not have
// enough space for a symbol (here: never)
std::cout << written << "-vs-" << symbol_size <<
" Could not get the whole repair symbol!\n";
return false;
}
// can we keep this symbol or do we randomly drop it?
float dropped = drop_rnd (rnd);
if (dropped <= drop_probability) {
continue; // start the cycle again
}
// good, the symbol was received.
++received_tot;
// add it to the vector of received symbols
symbol_id tmp_id = (*repair_sym_it).id();
received.emplace_back (tmp_id, std::move(repair_sym_data));
}
if (repair_sym_it == enc.end_repair (enc.max_repair())) {
// we dropped waaaay too many symbols!
// should never happen in real life. it means that we do not
// have enough repair symbols.
// at this point you can actually start to retransmit the
// repair symbols from enc.begin_repair(), but we don't care in
// this example
std::cout << "Maybe losing " << drop_probability << "% is too much?\n";
return false;
}
}
// Now we all the source and repair symbols are in "received".
// we will use those to start decoding:
// define "Decoder_type" to write less afterwards
using Decoder_type = RaptorQ::Decoder<
typename std::vector<uint8_t>::iterator,
typename std::vector<uint8_t>::iterator>;
Decoder_type dec (block, symbol_size, Decoder_type::Report::COMPLETE);
// "Decoder_type::Report::COMPLETE" means that the decoder will not
// give us any output until we have decoded all the data.
// there are modes to extract the data symbol by symbol in an ordered
// an unordered fashion, but let's keep this simple.
// we will store the output of the decoder here:
// note: the output need to have at least "mysize" bytes, and
// we fill it with zeros
std::vector<uint8_t> output (mysize, 0);
// now push every received symbol into the decoder
for (auto &rec_sym : received) {
// as a reminder:
// rec_sym.first = symbol_id (uint32_t)
// rec_sym.second = std::vector<uint8_t> symbol_data
symbol_id tmp_id = rec_sym.first;
auto it = rec_sym.second.begin();
auto err = dec.add_symbol (it, rec_sym.second.end(), tmp_id);
if (err != RaptorQ::Error::NONE && err != RaptorQ::Error::NOT_NEEDED) {
// When you add a symbol, you can get:
// NONE: no error
// NOT_NEEDED: libRaptorQ ignored it because everything is
// already decoded
// INITIALIZATION: wrong parameters to the decoder contructor
// WRONG_INPUT: not enough data on the symbol?
// some_other_error: errors in the library
std::cout << "error adding?\n";
return false;
}
}
// by now we now there will be no more input, so we tell this to the
// decoder. You can skip this call, but if the decoder does not have
// enough data it sill wait forever (or until you call .stop())
dec.end_of_input (RaptorQ::Fill_With_Zeros::NO);
// optional if you want partial decoding without using the repair
// symbols
// std::vector<bool> symbols_bitmask = dec.end_of_input (
// RaptorQ::Fill_With_Zeros::YES);
// decode, and do not return until the computation is finished.
auto res = dec.wait_sync();
if (res.error != RaptorQ::Error::NONE) {
std::cout << "Couldn't decode.\n";
return false;
}
// now save the decoded data in our output
size_t decode_from_byte = 0;
size_t skip_bytes_at_begining_of_output = 0;
auto out_it = output.begin();
auto decoded = dec.decode_bytes (out_it, output.end(), decode_from_byte,
skip_bytes_at_begining_of_output);
// "decode_from_byte" can be used to have only a part of the output.
// it can be used in advanced setups where you ask only a part
// of the block at a time.
// "skip_bytes_at_begining_of_output" is used when dealing with containers
// which size does not align with the output. For really advanced usage only
// Both should be zero for most setups.
if (decoded.written != mysize) {
if (decoded.written == 0) {
// we were really unlucky and the RQ algorithm needed
// more symbols!
std::cout << "Couldn't decode, RaptorQ Algorithm failure. "
"Can't Retry.\n";
} else {
// probably a library error
std::cout << "Partial Decoding? This should not have happened: " <<
decoded.written << " vs " << mysize << "\n";
}
return false;
} else {
std::cout << "Decoded: " << mysize << "\n";
}
// byte-wise check: did we actually decode everything the right way?
for (uint64_t i = 0; i < mysize; ++i) {
if (input[i] != output[i]) {
// this is a bug in the library, please report
std::cout << "The output does not correspond to the input!\n";
return false;
}
}
return true;
}
int main (void)
{
// get a random number generator
std::mt19937_64 rnd;
std::ifstream rand("/dev/urandom");
uint64_t seed = 0;
rand.read (reinterpret_cast<char *> (&seed), sizeof(seed));
rand.close ();
rnd.seed (seed);
// keep some computation in memory. If you use only one block size it
// will make things faster on bigger block size.
// allocate 5Mb
RaptorQ__v1::local_cache_size (5000000);
// for our test, we use an input of random size, between 100 and 10.000
// bytes.
std::uniform_int_distribution<uint32_t> distr(100, 10000);
uint32_t input_size = distr (rnd);
if (!test_rq (input_size, rnd, 20.0, 4))
return -1;
std::cout << "The example completed successfully\n";
return 0;
}