Skip to content
Commits on Source (147)
Expand all Hide whitespace changes
Inline Side-by-side
.gitignore
View file @ a394b22a
doc/libRaptorQ.aux /doc/*.aux
doc/libRaptorQ.bbl /doc/*.bbl
doc/libRaptorQ.blg /doc/*.blg
doc/libRaptorQ.idx /doc/*.idx
doc/libRaptorQ.ilg /doc/*.ilg
doc/libRaptorQ.ind /doc/*.ind
doc/libRaptorQ.log /doc/*.log
doc/libRaptorQ.out /doc/*.out
doc/libRaptorQ.pdf /doc/*.pdf
doc/libRaptorQ.synctex.gz /doc/*.synctex.gz
doc/libRaptorQ.toc /doc/*.toc
CMakeFiles/ /CMakeFiles/
CMakeCache.txt /CMakeCache.txt
CMakeLists.txt.user /CMakeLists.txt.user
cmake_install.cmake /cmake_install.cmake
Makefile /Makefile
*~ *~
build* /build*
.gitlab-ci.yml
View file @ a394b22a
...@@ -5,12 +5,12 @@ stages: ...@@ -5,12 +5,12 @@ stages:
job1: job1:
stage: makedoc stage: makedoc
script: mkdir build; cd build; cmake -DCMAKE_BUILD_TYPE=Release ../; make docs; cp doc/libRaptorQ.pdf ~/ script: mkdir build; cd build; cmake -DCMAKE_BUILD_TYPE=Release ../; make docs; cp doc/libRaptorQ-RFC6330.pdf ~/; cp doc/libRaptorQ-RaptorQ.pdf ~/
job2: job2:
stage: deploydoc stage: deploydoc
script: git diff-tree --no-commit-id --name-only -r HEAD | grep ^doc/ || exit 0; cd /var/lib/gitlab-runner/libRaptorQ.wiki; git pull; cp ~/libRaptorQ.pdf .; git add libRaptorQ.pdf; git commit -m '(gitlab-runner) updated pdf'; git push; script: git diff-tree --no-commit-id --name-only -r HEAD | grep ^doc/ || exit 0; cd /var/lib/gitlab-runner/libRaptorQ.wiki; git pull; cp ~/libRaptorQ-RFC6330.pdf .; cp ~/libRaptorQ-RaptorQ.pdf .; git add libRaptorQ-RFC6330.pdf; git add libRaptorQ-RaptorQ.pdf; git commit -m '(gitlab-runner) updated pdf'; git push;
job3: job3:
stage: build stage: build
......
CHANGELOG
View file @ a394b22a
v0.2.0 (prealpha): v1.0.0 (beta):
* C++98 compatibility (untested)
* we are now a header-only library that can be compiled and linked to C/C++98, with RFC and RAW API
* RAW API block size is now typesafe.
* RAW API does not use the interleaving code.
* API reworked to easy manage old/new API in the future
* new, simpler RAW API (single-block) * new, simpler RAW API (single-block)
* computations can be stopped
* precomputation caching * precomputation caching
* cached result can be compressed (raw/LZ4) * cached result can be compressed (raw/LZ4)
* RFC API: global multithreading instead of per-class * RFC API: global multithreading instead of per-class
* computations can be stopped * RFC compliance
v0.1.8 (unreleased): * CLI tool for raw API
* OSX/BSD support
v0.1.10 (unreleased):
* backported MAC/FreeBSD support (and incomplete windows)
* backported RFC compliance
v0.1.9:
* remember to bump up version in the cmake -.-'
* squashed warnings for gcc & clang in 32 bit mode.
v0.1.8:
* someone forgot to update the changelog :p * someone forgot to update the changelog :p
* was bugfixing only, no new feature * was bugfixing only, no new feature
* fixed initializing encoder/decoder with more than 256 blocks * fixed initializing encoder/decoder with more than 256 blocks
......
CMakeLists.txt
View file @ a394b22a
This diff is collapsed.
CONTRIBUTING.md
View file @ a394b22a
...@@ -66,7 +66,7 @@ developer accepted the CLA and is the author of the commit. ...@@ -66,7 +66,7 @@ developer accepted the CLA and is the author of the commit.
* Always sign your commits with the "-s" option, to add the "Signed-off-by" line in the commit message * Always sign your commits with the "-s" option, to add the "Signed-off-by" line in the commit message
* Only the main developers are required to GPG-sign the commits. * Only the main developers are required to GPG-sign the commits.
* Run the tests to make sure you have not broken something by mistake * Run the tests to make sure you have not broken something by mistake
* Please keep commits short and to the point. Nobody wants to review 2000+ lines of code that touch everything.
## Documentation ## Documentation
......
README.md
View file @ a394b22a
...@@ -2,11 +2,15 @@ ...@@ -2,11 +2,15 @@
[![build status](https://www.fenrirproject.org/Luker/libRaptorQ/badges/master/build.svg)](https://www.fenrirproject.org/Luker/libRaptorQ/commits/master) [![build status](https://www.fenrirproject.org/Luker/libRaptorQ/badges/master/build.svg)](https://www.fenrirproject.org/Luker/libRaptorQ/commits/master)
[![Donate](https://img.shields.io/badge/Donate-PayPal-blue.svg)](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=79L8P7TXWFQK4)
[![Donate](https://img.shields.io/badge/Donate-Patreon-orange.svg)](https://patreon.com/Luker)
[Homepage](https://www.fenrirproject.org/Luker/libRaptorQ/wikis/home) [Homepage](https://www.fenrirproject.org/Luker/libRaptorQ/wikis/home)
stable release: **v0.1.7**
current release: **v0.2-prealpha** old release: **v0.1.10**
**CURRENTLY BREAKING APIs, stick with the stable release** > Note: v0.1.10 is not compatible with prefivous versions, which were not RFC compliant!
current release: **v1.0.0-rc2** (use this in new projects)
RaptorQ is a **Forward Error Correction** algorithm designed to deliver your data RaptorQ is a **Forward Error Correction** algorithm designed to deliver your data
efficiently and without retransmissions for lost packets. efficiently and without retransmissions for lost packets.
...@@ -21,17 +25,30 @@ code in this category. ...@@ -21,17 +25,30 @@ code in this category.
**libRaptorQ** implements **RFC6330**, which specifies the RaptorQ algorithm. **libRaptorQ** implements **RFC6330**, which specifies the RaptorQ algorithm.
Since the RFC is really complex, there is also a simpler (and maybe slightly faster)
RAW API that you can use. The RFC API is not recommended due to the RFC complexity.
libRaptorQ is entirely written in *C++11*, and uses eigen to handle matrix libRaptorQ is a header-only library written in *C++11*, and uses eigen to handle matrix
manipulation. manipulation.
Although header-only, the library can be compiled to create shared and static libraries
with both C and C++98 compatibility.
Currently it's only been tested under Linux, but should work well under Currently it's only been tested under Linux, but should work well under
*BSD and MacOSX, too. *BSD and MacOSX, too.
Windows _should_ work too, provided you use a complier that understands C++11 Unfortunately Windows is not supported. support for <future> is broken,
(hint: you might have some problems with VisualC++, but that's not a C++ so you can not compile it with Visual Studio 2015.
compiler anyway)
## Features
- RFC6330 API (complex, not recommended)
- RAW API (simpler, recommended)
- multi language:
- header only C++11
- compiled, C
- compiled, C++98
- cached precomputations (configurable)
## Developers ## ## Developers ##
...@@ -58,45 +75,23 @@ you can also get it from our main server: ...@@ -58,45 +75,23 @@ you can also get it from our main server:
Once you have cloned it, it's always a good thing to check the repository gpg Once you have cloned it, it's always a good thing to check the repository gpg
signatures, so you can import my key with: signatures, so you can import my key with:
new long term key: **7393 DAD2 55BE B575 1DBD A04A B11C D823 BA27 8C85**
``$ gpg --keyserver pgp.mit.edu --recv-key 7393DAD255BEB5751DBDA04AB11CD823BA278C85``
2016 key: 2016 key:
``$ gpg --keyserver pgp.mit.edu --recv-key F61F6137`` ``$ gpg --keyserver pgp.mit.edu --recv-key F61F6137``
2015 key: 2015 key:
``$ gpg --keyserver pgp.mit.edu --recv-key D42DDF0A`` ``$ gpg --keyserver pgp.mit.edu --recv-key D42DDF0A``
please check the full fingerprint, it should be like this: please check the full fingerprint.
```bash
$ gpg2 --fingerprint F61F6137
pub rsa2048/F61F6137 2016-01-11 [expires: 2017-01-25]
Key fingerprint = 95EB 6FA4 03D7 29A7 13C2 FC9F F85A DA1C F61F 6137
uid [ultimate] Luca Fulchir (2016 key) <luker@fenrirproject.org>
full 2015 fingerprint:
Key fingerprint = AB35 E45F 5CA5 E35B 8B55 818F 0157 D133 D42D DF0A
```
Now you have the source, and the key, it's enough to check the signature of the Now you have the source, and the key, it's enough to check the signature of the
last commit: last commit:
``$ git log -n 1 --show-signature`` ``$ git log -n 1 --show-signature``
The important part is that you get something like this: As long as you got the right key, and you find the "gpg: Goog signature",
```bash
gpg: Signature made Mon 11 Jan 2016 14:19:21 CET using RSA key ID F61F6137
gpg: Good signature from "Luca Fulchir (2016 key) <luker@fenrirproject.org>"
[unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 95EB 6FA4 03D7 29A7 13C2 FC9F F85A DA1C F61F 6137
Author: Luca Fulchir <luker@fenrirproject.org>
```
And as long as you got the right key, and you find the "gpg: Goog signature",
you can be sure you have the right code. you can be sure you have the right code.
TDB: The repository is using the maintainer private key for now,
it will start using a dedicated key in the (near) future.
## Install ## ## Install ##
...@@ -149,25 +144,43 @@ break deterministic builds, so check the cmake warnings._ ...@@ -149,25 +144,43 @@ break deterministic builds, so check the cmake warnings._
You can customize the CMake build with the following variables: You can customize the CMake build with the following variables:
``` ```
PROFILING ON/OFF: Default:ON. Activate or deactivate profiling. PROFILING ON/OFF: Default:ON. Activate or deactivate profiling.
Profiling compiles everything, then runs a test to see Profiling compiles everything, then runs a test to see
which code paths are more used. Then it recompiles everything which code paths are more used. Then it recompiles everything
but optimizing for for those code paths. but optimizing for for those code paths.
Only for gcc/clang. Only for gcc/clang.
LTO ON/OFF: Default:ON. Activate or deactivate Link time Optimization LTO ON/OFF: Default:ON. Activate or deactivate Link time Optimization
Makes the library smaller and better optimized. Makes the library smaller and better optimized.
Only for gcc/clang. Only for gcc/clang.
CLANG_STDLIB ON/OFF: Default:OFF. use clang's libc++ CLANG_STDLIB ON/OFF: Default:OFF. use clang's libc++
Note: only clang can use its standard library Note: only clang can use its standard library
CMAKE_C_COMPILER gcc, clang... USE_LZ4 ON/OFF: use the lz4 compression for caching precomputations.
CMAKE_CXX_COMPILER choose between g++ or clang++. Default: ON
CMAKE_BUILD_TYPE Debug,MinSizeRel,Release,RelWithDebInfo CLI ON/OFF Build Command Line Interface tools.
CMAKE_C_COMPILER gcc, clang...
CMAKE_CXX_COMPILER choose between g++ or clang++.
RQ_LINKER gold/ld/bsd Choose your linker. Default:autodetect.
CMAKE_BUILD_TYPE Debug,MinSizeRel,Release,RelWithDebInfo
CMAKE_INSTALL_PREFIX Default: /usr/local CMAKE_INSTALL_PREFIX Default: /usr/local
``` ```
## Using libRaptorQ ## ## Using libRaptorQ ##
C and C++11 interfaces are available. For the C++11, header-only version, you can include:
- "RaptorQ/RaptorQ_v1_hdr.hpp"
- "RaptorQ/RFC6330_v1_hdr.hpp"
For the linked, C+98/C++11 API:
- "RaptorQ/RaptorQ_v1.hpp"
- "RaptorQ/RFC6330_v1.hpp"
For the linked, C API:
- "RaptorQ/RaptorQ.h"
- "RaptorQ/RFC6330.h"
The C++ api is completely in sync between the linked and header-only version,
so you can switch between the two just by changing the included header file.
You can compile a PDF of the documentation by doing: You can compile a PDF of the documentation by doing:
``$ make docs`` ``$ make docs``
......
cmake/FindRQ_LZ4.cmake
View file @ a394b22a
...@@ -19,38 +19,54 @@ ...@@ -19,38 +19,54 @@
# #
FIND_PATH(RQ_LZ4_INCLUDE_DIR FIND_PATH(RQ_LZ4_INCLUDE_DIR
NAMES lz4.h NAMES lz4.h
PATH_SUFFIXES include/ include/lz4/ lz4/ PATH_SUFFIXES include/ include/lz4/ lz4/
PATHS PATHS
${LZ4_ROOT} ${LZ4_ROOT}
$ENV{LZ4_ROOT} $ENV{LZ4_ROOT}
/usr/ /usr/
${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib /usr/local/
${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib
)
FIND_LIBRARY(RQ_LZ4_LIB
NAMES lz4
PATH_SUFFIXES lib/
PATHS
${LZ4_ROOT}
$ENV{LZ4_ROOT}
/usr/
/usr/local/
) )
IF(RQ_LZ4_INCLUDE_DIR) IF(RQ_LZ4_INCLUDE_DIR)
SET(RQ_LZ4_FOUND TRUE) SET(RQ_LZ4_FOUND TRUE)
ELSE(RQ_LZ4_INCLUDE_DIR) ELSE(RQ_LZ4_INCLUDE_DIR)
SET(RQ_LZ4_FOUND FALSE) SET(RQ_LZ4_FOUND FALSE)
ENDIF(RQ_LZ4_INCLUDE_DIR) ENDIF(RQ_LZ4_INCLUDE_DIR)
IF(RQ_LZ4_FOUND) IF(RQ_LZ4_FOUND)
MESSAGE(STATUS "Found lz4 in ${RQ_LZ4_INCLUDE_DIR}") IF(RQ_LZ4_USE_OWN)
IF(RQ_LZ4_INCLUDE_DIR MATCHES "${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib") # force our own lz4 libary
MESSAGE(WARNING "We will build our own lz4 library and statically link it.") SET(RQ_LZ4_INCLUDE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib)
SET(RQ_BUILD_LZ4 TRUE) ENDIF()
ENDIF() MESSAGE(STATUS "Found lz4 in ${RQ_LZ4_INCLUDE_DIR}")
# we need both headers and library, or we use our own
IF(RQ_LZ4_INCLUDE_DIR MATCHES "${CMAKE_CURRENT_SOURCE_DIR}/external/lz4/lib" OR RQ_LZ4_LIB MATCHES "RQ_LZ4_LIB-NOTFOUND")
MESSAGE(WARNING "We will build our own lz4 library and statically link it.")
SET(RQ_BUILD_LZ4 TRUE)
ENDIF()
ELSE(RQ_LZ4_FOUND) ELSE(RQ_LZ4_FOUND)
IF(RQ_LZ4_FIND_REQUIRED) IF(RQ_LZ4_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"lz4\" library." MESSAGE(FATAL_ERROR "Could not find \"lz4\" library."
" Please install the \"lz4\" library" " Please install the \"lz4\" library"
" or at least run:\n" " or at least run:\n"
" cd ${CMAKE_CURRENT_SOURCE_DIR}\n" " cd ${CMAKE_CURRENT_SOURCE_DIR}\n"
" git submodule init\n" " git submodule init\n"
" git submodule update\n") " git submodule update\n")
ENDIF(RQ_LZ4_FIND_REQUIRED) ENDIF(RQ_LZ4_FIND_REQUIRED)
ENDIF(RQ_LZ4_FOUND) ENDIF(RQ_LZ4_FOUND)
MARK_AS_ADVANCED( MARK_AS_ADVANCED(
RQ_LZ4_INCLUDE_DIR RQ_LZ4_INCLUDE_DIR
) )
cmake/Findeigen.cmake
View file @ a394b22a
...@@ -19,30 +19,31 @@ ...@@ -19,30 +19,31 @@
# #
FIND_PATH(EIGEN3_INCLUDE_DIR FIND_PATH(EIGEN3_INCLUDE_DIR
NAMES Eigen NAMES Eigen
PATH_SUFFIXES include/ include/eigen3/ eigen3/ PATH_SUFFIXES include/ include/eigen3/ eigen3/
PATHS PATHS
${EIGEN3_ROOT} ${EIGEN3_ROOT}
$ENV{EIGEN3_ROOT} $ENV{EIGEN3_ROOT}
/usr/ /usr/
${PROJECT_SOURCE_DIR}/external/eigen3 /usr/local/
${PROJECT_SOURCE_DIR}/external/eigen3
) )
IF(EIGEN3_INCLUDE_DIR) IF(EIGEN3_INCLUDE_DIR)
SET(EIGEN3_FOUND TRUE) SET(EIGEN3_FOUND TRUE)
ELSE(EIGEN3_INCLUDE_DIR) ELSE(EIGEN3_INCLUDE_DIR)
SET(EIGEN3_FOUND FALSE) SET(EIGEN3_FOUND FALSE)
ENDIF(EIGEN3_INCLUDE_DIR) ENDIF(EIGEN3_INCLUDE_DIR)
IF(EIGEN3_FOUND) IF(EIGEN3_FOUND)
MESSAGE(STATUS "Found eigen3 in ${EIGEN3_INCLUDE_DIR}") MESSAGE(STATUS "Found eigen3 in ${EIGEN3_INCLUDE_DIR}")
ELSE(EIGEN3_FOUND) ELSE(EIGEN3_FOUND)
IF(EIGEN3_FIND_REQUIRED) IF(EIGEN3_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"eigen3\" library") MESSAGE(FATAL_ERROR "Could not find \"eigen3\" library")
ENDIF(EIGEN3_FIND_REQUIRED) ENDIF(EIGEN3_FIND_REQUIRED)
ENDIF(EIGEN3_FOUND) ENDIF(EIGEN3_FOUND)
MARK_AS_ADVANCED( MARK_AS_ADVANCED(
EIGEN3_INCLUDE_DIR EIGEN3_INCLUDE_DIR
) )
cmake/Findgit.cmake
View file @ a394b22a
...@@ -18,29 +18,29 @@ ...@@ -18,29 +18,29 @@
# along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>. # along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
# #
FIND_PROGRAM(GIT_EXECUTABLE git FIND_PROGRAM(GIT_EXECUTABLE git
PATHS PATHS
${PATH} ${PATH}
"C:/Program Files/Git/bin" "C:/Program Files/Git/bin"
"C:/Program Files (x86)/Git/bin" "C:/Program Files (x86)/Git/bin"
DOC "git command line client" DOC "git command line client"
) )
IF(GIT_EXECUTABLE) IF(GIT_EXECUTABLE)
SET(GIT_FOUND TRUE) SET(GIT_FOUND TRUE)
ELSE(GIT_EXECUTABLE) ELSE(GIT_EXECUTABLE)
SET(GIT_FOUND FALSE) SET(GIT_FOUND FALSE)
ENDIF(GIT_EXECUTABLE) ENDIF(GIT_EXECUTABLE)
IF(GIT_FOUND) IF(GIT_FOUND)
MESSAGE(STATUS "Found git in ${GIT_EXECUTABLE}") MESSAGE(STATUS "Found git in ${GIT_EXECUTABLE}")
ELSE(GIT_FOUND) ELSE(GIT_FOUND)
IF(GIT_REQUIRED) IF(GIT_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find \"git\" command") MESSAGE(FATAL_ERROR "Could not find \"git\" command")
ENDIF(GIT_REQUIRED) ENDIF(GIT_REQUIRED)
ENDIF(GIT_FOUND) ENDIF(GIT_FOUND)
MARK_AS_ADVANCED( MARK_AS_ADVANCED(
GIT_EXECUTABLE GIT_EXECUTABLE
) )
......
cmake/Findstd_clang.cmake deleted 100644 → 0
View file @ 701c0225
#
# Copyright (c) 2015, Luca Fulchir<luca@fulchir.it>, All rights reserved.
#
# This file is part of libRaptorQ.
#
# libRaptorQ is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation, either version 3
# of the License, or (at your option) any later version.
#
# libRaptorQ is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# and a copy of the GNU Lesser General Public License
# along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
#
FIND_PATH(CLANG_STD_INCLUDE_DIR
NAMES unordered_map
PATH_SUFFIXES include/c++ include/c++/v1
PATHS "${PROJECT_SOURCE_DIR}/../libc++/"
${CLANG_STD_ROOT}
$ENV{CLANG_STD_ROOT}
/usr/local/
/usr/
)
message("Found clang stdandard library at: ${CLANG_STD_INCLUDE_DIR}.")
IF(CLANG_STD_INCLUDE_DIR)
SET(CLANG_STD_FOUND TRUE)
ELSE(CLANG_STD_INCLUDE_DIR)
SET(CLANG_STD_FOUND FALSE)
ENDIF(CLANG_STD_INCLUDE_DIR)
IF(CLANG_STD_FOUND)
MESSAGE(STATUS "Found clang standard library in ${CLANG_STD_INCLUDE_DIR}")
ELSE(CLANG_STD_FOUND)
IF(CLANG_STD_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find the clang standard library")
ENDIF(CLANG_STD_FIND_REQUIRED)
ENDIF(CLANG_STD_FOUND)
MARK_AS_ADVANCED(
CLANG_STD_INCLUDE_DIR
)
cmake/Findstd_gcc.cmake deleted 100644 → 0
View file @ 701c0225
#
# Copyright (c) 2015, Luca Fulchir<luca@fulchir.it>, All rights reserved.
#
# This file is part of libRaptorQ.
#
# libRaptorQ is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation, either version 3
# of the License, or (at your option) any later version.
#
# libRaptorQ is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# and a copy of the GNU Lesser General Public License
# along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
#
file(GLOB sub-dir /usr/include/c++/[0-9].[0-9]*.[0-9]*)
foreach(dir ${sub-dir})
if(IS_DIRECTORY ${dir})
set(GCC_PATH_SUFFIXES ${GCC_PATH_SUFFIXES};${dir})
endif()
endforeach()
FIND_PATH(GCC_STD_INCLUDE_DIR
NAMES unordered_map
PATH_SUFFIXES / ${GCC_PATH_SUFFIXES}
PATHS "${PROJECT_SOURCE_DIR}/../stdlib/"
${GCC_STD_ROOT}
$ENV{GCC_STD_ROOT}
/
)
message("Found gcc stdandard library at: ${GCC_STD_INCLUDE_DIR}.")
IF(GCC_STD_INCLUDE_DIR)
SET(GCC_STD_FOUND TRUE)
ELSE(GCC_STD_INCLUDE_DIR)
SET(GCC_STD_FOUND FALSE)
ENDIF(GCC_STD_INCLUDE_DIR)
IF(GCC_STD_FOUND)
MESSAGE(STATUS "Found gcc standard library in ${GCC_STD_INCLUDE_DIR}")
ELSE(GCC_STD_FOUND)
IF(GCC_STD_FIND_REQUIRED)
MESSAGE(FATAL_ERROR "Could not find the gcc standard library")
ENDIF(GCC_STD_FIND_REQUIRED)
ENDIF(GCC_STD_FOUND)
MARK_AS_ADVANCED(
GCC_STD_INCLUDE_DIR
)
cmake/compiler_flags.cmake
View file @ a394b22a
...@@ -21,6 +21,10 @@ ...@@ -21,6 +21,10 @@
include(CheckCCompilerFlag) include(CheckCCompilerFlag)
include(CheckCXXCompilerFlag) include(CheckCXXCompilerFlag)
#string of multiple flags to list
string(REPLACE " " ";" RQ_C_FLAGS "${CMAKE_C_FLAGS}")
string(REPLACE " " ";" RQ_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
############################################## ##############################################
#### check for UBSAN sanitizing support ###### #### check for UBSAN sanitizing support ######
############################################## ##############################################
...@@ -29,11 +33,13 @@ if(CMAKE_BUILD_TYPE MATCHES "Debug") ...@@ -29,11 +33,13 @@ if(CMAKE_BUILD_TYPE MATCHES "Debug")
check_cxx_compiler_flag("-fsanitize=undefined" RQ_FLAG_CXX_UBSAN) check_cxx_compiler_flag("-fsanitize=undefined" RQ_FLAG_CXX_UBSAN)
if (RQ_FLAG_C_UBSAN AND RQ_FLAG_CXX_UBSAN) if (RQ_FLAG_C_UBSAN AND RQ_FLAG_CXX_UBSAN)
set(RQ_ENABLE_UBSAN TRUE) set(RQ_ENABLE_UBSAN TRUE)
set(RQ_UBSAN ubsan)
message(STATUS "UBSAN sanitizing support enabled") message(STATUS "UBSAN sanitizing support enabled")
else() else()
message(STATUS "UBSAN sanitizing support disabled") message(STATUS "UBSAN sanitizing support disabled")
endif() endif()
add_definitions(-DRQ_DEBUG=true)
else()
add_definitions(-DRQ_DEBUG=false)
endif() endif()
################### ###################
...@@ -41,8 +47,19 @@ endif() ...@@ -41,8 +47,19 @@ endif()
################### ###################
#gnu options #gnu options
set(RQ_GNU_C_OPTIONS ${RQ_DETERMINISTIC} -std=c11 -Wno-unknown-pragmas -Wall -pedantic -Wno-padded -fstack-protector-all -fstrict-aliasing -fwrapv -fvisibility=hidden ) set(RQ_GNU_C_OPTIONS ${RQ_DETERMINISTIC} -std=c11
set(RQ_GNU_CXX_OPTIONS ${RQ_DETERMINISTIC} -std=c++11 -fno-rtti -fno-exceptions -Wno-unknown-pragmas -Wall -pedantic -Wno-padded -Wno-unknown-pragmas -fstack-protector-all -fstrict-aliasing -fwrapv -fvisibility=hidden -fvisibility-inlines-hidden ) -Wno-unknown-pragmas -Wall -Wextra -pedantic -Wno-padded
-fstack-protector-all -fstrict-aliasing -fwrapv -fvisibility=hidden
-Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict
-Wnull-dereference -Wdouble-promotion -Wshadow -Wformat=2)
set(RQ_GNU_CXX_OPTIONS ${RQ_DETERMINISTIC} -std=c++11
-fno-rtti -fno-exceptions -Wno-unknown-pragmas -Wall -Wextra
-pedantic -Wno-padded -Wno-unknown-pragmas -fstack-protector-all
-fstrict-aliasing -fwrapv -fvisibility=hidden -fvisibility-inlines-hidden
-Wduplicated-cond -Wduplicated-branches -Wlogical-op -Wrestrict
-Wnull-dereference -Wuseless-cast -Wdouble-promotion -Wshadow -Wformat=2
-Wno-duplicated-branches)
# GCC internal compiler errors with: # GCC internal compiler errors with:
# -fsanitize=undefined # -fsanitize=undefined
...@@ -54,14 +71,24 @@ set(RQ_GNU_C_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero ...@@ -54,14 +71,24 @@ set(RQ_GNU_C_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero
set(RQ_GNU_CXX_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=vla-bound -fsanitize=return -fisolate-erroneous-paths-dereference -fisolate-erroneous-paths-attribute ) set(RQ_GNU_CXX_DEBUG_SANITIZE -fsanitize=shift -fsanitize=integer-divide-by-zero -fsanitize=vla-bound -fsanitize=return -fisolate-erroneous-paths-dereference -fisolate-erroneous-paths-attribute )
set(RQ_GNU_C_MINSIZEREL -Os ) set(RQ_GNU_C_MINSIZEREL -Os )
set(RQ_GNU_CXX_MINSIZEREL -Os ) set(RQ_GNU_CXX_MINSIZEREL -Os )
set(RQ_GNU_C_RELEASE -Ofast -DNDEBUG -fwrapv ) set(RQ_GNU_C_RELEASE -Ofast -DNDEBUG -fwrapv -ftree-loop-distribution -funroll-loops )
set(RQ_GNU_CXX_RELEASE -Ofast -DNDEBUG -fwrapv ) set(RQ_GNU_CXX_RELEASE -Ofast -DNDEBUG -fwrapv -ftree-loop-distribution -funroll-loops )
set(RQ_GNU_C_RELWITHDEBINFO ${RQ_GNU_C_RELEASE} -g ) set(RQ_GNU_C_RELWITHDEBINFO -g -Ofast -fwrapv -ftree-loop-distribution -funroll-loops )
set(RQ_GNU_CXX_RELWITHDEBINFO ${RQ_GNU_CXX_RELEASE} -g ) set(RQ_GNU_CXX_RELWITHDEBINFO -g -Ofast -fwrapv -ftree-loop-distribution -funroll-loops )
# clang options # clang options
set(RQ_CLANG_C_OPTIONS ${RQ_DETERMINISTIC} -std=c11 -Wall -pedantic -Weverything -Wno-padded -fstack-protector-all -fstrict-aliasing -Wformat -Wformat-security -Wno-disabled-macro-expansion -fvisibility=hidden -fvisibility-inlines-hidden) set(RQ_CLANG_C_OPTIONS ${RQ_DETERMINISTIC} -std=c11
set(RQ_CLANG_CXX_OPTIONS ${RQ_STDLIB_FLAG} ${RQ_DETERMINISTIC} -std=c++11 -fno-rtti -fno-exceptions -Wall -pedantic -Weverything -Wno-c++98-compat-pedantic -Wno-c++98-compat -Wno-padded -Wno-unknown-pragmas -fstack-protector-all -fstrict-aliasing -Wformat -Wformat-security -fvisibility=hidden -fvisibility-inlines-hidden) -fno-math-errno -Wall -Wextra -pedantic -Weverything -Wno-padded
-fstack-protector-all -fstrict-aliasing -Wformat -Wformat-security
-Wno-disabled-macro-expansion -fvisibility=hidden -fvisibility-inlines-hidden
-Wdouble-promotion -Wshadow -Wformat=2 -Wnull-dereference)
set(RQ_CLANG_CXX_OPTIONS ${RQ_STDLIB_FLAG} ${RQ_DETERMINISTIC}
-std=c++11 -fno-rtti -fno-exceptions -fno-math-errno
-Wall -pedantic -Weverything -Wno-c++98-compat-pedantic -Wno-c++98-compat
-Wno-padded -Wno-unknown-pragmas -fstack-protector-all -fstrict-aliasing
-Wformat -Wformat-security -fvisibility=hidden -fvisibility-inlines-hidden
-Wno-documentation -Wno-documentation-unknown-command
-Wdouble-promotion -Wshadow -Wformat=2 -Wnull-dereference)
set(RQ_CLANG_C_DEBUG -O0 -g ) set(RQ_CLANG_C_DEBUG -O0 -g )
set(RQ_CLANG_CXX_DEBUG -O0 -g ) set(RQ_CLANG_CXX_DEBUG -O0 -g )
...@@ -71,12 +98,12 @@ set(RQ_CLANG_C_MINSIZEREL -Os ) ...@@ -71,12 +98,12 @@ set(RQ_CLANG_C_MINSIZEREL -Os )
set(RQ_CLANG_CXX_MINSIZEREL -Os ) set(RQ_CLANG_CXX_MINSIZEREL -Os )
set(RQ_CLANG_C_RELEASE -Ofast -DNDEBUG -fwrapv ) set(RQ_CLANG_C_RELEASE -Ofast -DNDEBUG -fwrapv )
set(RQ_CLANG_CXX_RELEASE -Ofast -DNDEBUG -fwrapv ) set(RQ_CLANG_CXX_RELEASE -Ofast -DNDEBUG -fwrapv )
set(RQ_CLANG_C_RELWITHDEBINFO ${RQ_CLANG_C_RELEASE} -g ) set(RQ_CLANG_C_RELWITHDEBINFO -g -Ofast -fwrapv )
set(RQ_CLANG_CXX_RELWITHDEBINFO ${RQ_CLANG_CXX_RELEASE} -g ) set(RQ_CLANG_CXX_RELWITHDEBINFO -g -Ofast -fwrapv )
# msvc flags. todo? # msvc flags. todo?
set(MSVC_C_OPTIONS ) set(RQ_MSVC_C_OPTIONS /wd4068)
set(MSVC_CXX_OPTIONS ) set(RQ_MSVC_CXX_OPTIONS /wd4068)
################### ###################
...@@ -84,11 +111,12 @@ set(MSVC_CXX_OPTIONS ) ...@@ -84,11 +111,12 @@ set(MSVC_CXX_OPTIONS )
################### ###################
# note: generator expansions do not work on "set(..)" # note: generator expansions do not work on "set(..)"
if(CMAKE_CXX_COMPILER_ID MATCHES "Clang") if(CMAKE_CXX_COMPILER_ID MATCHES "Clang")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_CLANG_CXX_OPTIONS}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_CXX_FLAGS} ${RQ_CLANG_CXX_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug") if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_DEBUG}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_DEBUG})
if (RQ_ENABLE_UBSAN) if (RQ_ENABLE_UBSAN)
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_DEBUG_SANITIZE}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_DEBUG_SANITIZE})
set(RQ_UBSAN ${RQ_CLANG_CXX_DEBUG_SANITIZE})
endif() endif()
elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel") elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_MINSIZEREL}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_MINSIZEREL})
...@@ -98,11 +126,12 @@ if(CMAKE_CXX_COMPILER_ID MATCHES "Clang") ...@@ -98,11 +126,12 @@ if(CMAKE_CXX_COMPILER_ID MATCHES "Clang")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_RELWITHDEBINFO}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_CLANG_CXX_RELWITHDEBINFO})
endif() endif()
elseif(CMAKE_CXX_COMPILER_ID MATCHES "GNU") elseif(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_GNU_CXX_OPTIONS}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_CXX_FLAGS} ${RQ_GNU_CXX_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug") if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_DEBUG}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_DEBUG})
if (RQ_ENABLE_UBSAN) if (RQ_ENABLE_UBSAN)
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_DEBUG_SANITIZE}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_DEBUG_SANITIZE})
set(RQ_UBSAN ${RQ_GNU_CXX_DEBUG_SANITIZE})
endif() endif()
elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel") elseif(CMAKE_BUILD_TYPE MATCHES "MinSizeRel")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_MINSIZEREL}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_MINSIZEREL})
...@@ -112,12 +141,12 @@ elseif(CMAKE_CXX_COMPILER_ID MATCHES "GNU") ...@@ -112,12 +141,12 @@ elseif(CMAKE_CXX_COMPILER_ID MATCHES "GNU")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_RELWITHDEBINFO}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_LIST_CXX_COMPILER_FLAGS} ${RQ_GNU_CXX_RELWITHDEBINFO})
endif() endif()
elseif(CMAKE_CXX_COMPILER_ID MATCHES "MSVC") elseif(CMAKE_CXX_COMPILER_ID MATCHES "MSVC")
set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_MSVC_CXX_OPTIONS}) set(RQ_LIST_CXX_COMPILER_FLAGS ${RQ_CXX_FLAGS} ${RQ_MSVC_CXX_OPTIONS})
endif() endif()
if(CMAKE_C_COMPILER_ID MATCHES "Clang") if(CMAKE_C_COMPILER_ID MATCHES "Clang")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_CLANG_C_OPTIONS}) set(RQ_LIST_C_COMPILER_FLAGS ${RQ_C_FLAGS} ${RQ_CLANG_C_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug") if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_DEBUG}) set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_DEBUG})
if (RQ_ENABLE_UBSAN) if (RQ_ENABLE_UBSAN)
...@@ -131,7 +160,7 @@ if(CMAKE_C_COMPILER_ID MATCHES "Clang") ...@@ -131,7 +160,7 @@ if(CMAKE_C_COMPILER_ID MATCHES "Clang")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_RELWITHDEBINFO}) set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_CLANG_C_RELWITHDEBINFO})
endif() endif()
elseif(CMAKE_C_COMPILER_ID MATCHES "GNU") elseif(CMAKE_C_COMPILER_ID MATCHES "GNU")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_GNU_C_OPTIONS}) set(RQ_LIST_C_COMPILER_FLAGS ${RQ_C_FLAGS} ${RQ_GNU_C_OPTIONS})
if(CMAKE_BUILD_TYPE MATCHES "Debug") if(CMAKE_BUILD_TYPE MATCHES "Debug")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_DEBUG}) set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_DEBUG})
if (RQ_ENABLE_UBSAN) if (RQ_ENABLE_UBSAN)
...@@ -145,7 +174,7 @@ elseif(CMAKE_C_COMPILER_ID MATCHES "GNU") ...@@ -145,7 +174,7 @@ elseif(CMAKE_C_COMPILER_ID MATCHES "GNU")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_RELWITHDEBINFO}) set(RQ_LIST_C_COMPILER_FLAGS ${RQ_LIST_C_COMPILER_FLAGS} ${RQ_GNU_C_RELWITHDEBINFO})
endif() endif()
elseif(CMAKE_C_COMPILER_ID MATCHES "MSVC") elseif(CMAKE_C_COMPILER_ID MATCHES "MSVC")
set(RQ_LIST_C_COMPILER_FLAGS ${RQ_MSVC_C_OPTIONS}) set(RQ_LIST_C_COMPILER_FLAGS ${RQ_C_FLAGS} ${RQ_MSVC_C_OPTIONS})
endif() endif()
...@@ -182,4 +211,3 @@ foreach(flag ${RQ_LIST_CXX_COMPILER_FLAGS}) ...@@ -182,4 +211,3 @@ foreach(flag ${RQ_LIST_CXX_COMPILER_FLAGS})
endif() endif()
endforeach() endforeach()
cmake/guess_architecture.cmake
View file @ a394b22a
# #
# Copyright (c) 2015, Luca Fulchir<luca@fulchir.it>, All rights reserved. # Copyright (c) 2015-2018, Luca Fulchir<luca@fulchir.it>, All rights reserved.
# #
# This file is part of libRaptorQ. # This file is part of libRaptorQ.
# #
...@@ -57,5 +57,3 @@ ELSE() ...@@ -57,5 +57,3 @@ ELSE()
ENDIF() ENDIF()
ENDIF() ENDIF()
doc/CMakeLists.txt
View file @ a394b22a
...@@ -23,17 +23,24 @@ CMAKE_MINIMUM_REQUIRED(VERSION 2.8) ...@@ -23,17 +23,24 @@ CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
INCLUDE(../cmake/UseLATEX.cmake) INCLUDE(../cmake/UseLATEX.cmake)
IF (LATEX_COMPILER AND PDFLATEX_COMPILER AND MAKEINDEX_COMPILER) IF (LATEX_COMPILER AND PDFLATEX_COMPILER AND MAKEINDEX_COMPILER)
ADD_LATEX_DOCUMENT(libRaptorQ.tex ADD_LATEX_DOCUMENT(libRaptorQ-RFC6330.tex
INPUTS GNU_FDL.tex INPUTS GNU_FDL.tex
USE_INDEX USE_INDEX
MANGLE_TARGET_NAMES MANGLE_TARGET_NAMES
NO_DEFAULT NO_DEFAULT
) )
ADD_LATEX_DOCUMENT(libRaptorQ-RaptorQ.tex
INPUTS GNU_FDL.tex
USE_INDEX
MANGLE_TARGET_NAMES
NO_DEFAULT
)
ADD_CUSTOM_TARGET(docs DEPENDS libRaptorQ_pdf) ADD_CUSTOM_TARGET(docs DEPENDS libRaptorQ-RFC6330_pdf libRaptorQ-RaptorQ_pdf)
ELSE() ELSE()
ADD_CUSTOM_TARGET(docs) ADD_CUSTOM_TARGET(docs)
ENDIF() ENDIF()
INSTALL(FILES ${CMAKE_BINARY_DIR}/doc/libRaptorQ.pdf DESTINATION share/doc/ OPTIONAL) INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/doc/libRaptorQ-RFC6330.pdf DESTINATION share/doc/ OPTIONAL)
INSTALL(FILES ${CMAKE_CURRENT_BINARY_DIR}/doc/libRaptorQ-RaptorQ.pdf DESTINATION share/doc/ OPTIONAL)
doc/libRaptorQ-RFC6330.tex 0 → 100644
View file @ a394b22a
This diff is collapsed.
examples/example_cpp_raw.cpp 0 → 100644
View file @ a394b22a
/*
* Copyright (c) 2015-2018, Luca Fulchir<luker@fenrirproject.org>,
* All rights reserved.
*
* This file is part of "libRaptorQ".
*
* libRaptorQ is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as
* published by the Free Software Foundation, either version 3
* of the License, or (at your option) any later version.
*
* libRaptorQ is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* and a copy of the GNU Lesser General Public License
* along with libRaptorQ. If not, see <http://www.gnu.org/licenses/>.
*/
#include "../src/RaptorQ/RaptorQ_v1_hdr.hpp"
#include <algorithm>
#include <fstream>
#include <iostream>
#include <limits>
#include <random>
#include <stdlib.h>
#include <vector>
// Demonstration of how to use the C++ RAW interface
// it's pretty simple, we generate some input,
// then encode, drop some packets (source and repair)
// and finally decode everything.
// rename the main namespace for ease of use
namespace RaptorQ = RaptorQ__v1;
// mysize is bytes.
bool test_rq (const uint32_t mysize, std::mt19937_64 &rnd,
float drop_probability,
const uint8_t overhead);
// the "overhead" variable tells us how many symbols more than the
// minimum we will generate. RaptorQ can not always decode a block,
// but there is a small probability that it will fail.
// More overhead => less probability of failure
// overhead 0 => 1% failures
// overhead 1 => 0.01% failures
// overhead 2 => 0.0001% failures
// etc... as you can see, it make little sense to work with more than 3-4
// overhead symbols, but at least one should be considered
bool test_rq (const uint32_t mysize, std::mt19937_64 &rnd,
float drop_probability,
const uint8_t overhead)
{
// the actual input.
std::vector<uint8_t> input;
input.reserve (mysize);
// initialize vector with random data
// distr should be "uint8_t". But visual studio does not like it, so
// we use uint16_t
// generate a random distribution between all values of uint8_t
std::uniform_int_distribution<int16_t> distr (0,
std::numeric_limits<uint8_t>::max());
// fill our input with random data
for (size_t idx = 0; idx < mysize; ++idx) {
input.push_back (static_cast<uint8_t> (distr(rnd)));
}
// the input will be divided in blocks of 4 bytes.
// it's a bit low, but this is just an example.
// NOTE: the symbol size must be a multiple of the container size.
// since sizeof(uint8_t) == 1 and 4 is a multiple of 1, we are safe.
const uint16_t symbol_size = 4; // bytes
// how many symbols do we need to encode all our input in a single block?
auto min_symbols = (input.size() * sizeof(uint8_t)) / symbol_size;
if ((input.size() * sizeof(uint8_t)) % symbol_size != 0)
++min_symbols;
// convert "symbols" to a typesafe equivalent, RaptorQ::Block_Size
// This is needed becouse not all numbers are valid block sizes, and this
// helps you choose the right block size
RaptorQ::Block_Size block = RaptorQ::Block_Size::Block_10;
for (auto blk : *RaptorQ::blocks) {
// RaptorQ::blocks is a pointer to an array, just scan it to find your
// block.
if (static_cast<uint16_t> (blk) >= min_symbols) {
block = blk;
break;
}
}
// now initialize the encoder.
// the input for the encoder is std::vector<uint8_t>
// the output for the encoder is std::vector<uint8_t>
// yes, you can have different types, but most of the time you will
// want to work with uint8_t
RaptorQ::Encoder<typename std::vector<uint8_t>::iterator,
typename std::vector<uint8_t>::iterator> enc (
block, symbol_size);
// give the input to the encoder. the encoder answers with the size of what
// it can use
if (enc.set_data (input.begin(), input.end()) != mysize) {
std::cout << "Could not give data to the encoder :(\n";
return false;
}
// actual symbols. you could just use static_cast<uint16_t> (blok)
// but this way you can actually query the encoder.
uint16_t _symbols = enc.symbols();
// print some stuff in output
std::cout << "Size: " << mysize << " symbols: " <<
static_cast<uint32_t> (_symbols) <<
" symbol size: " <<
static_cast<int32_t>(enc.symbol_size()) << "\n";
// RQ need to do its magic on the input before you can ask the symbols.
// multiple ways to do this are available.
// The simplest is to run the computation and block everything until
// the work has been done. Not a problem for small sizes (<200),
// but big sizes will take **a lot** of time, consider running this with the
// asynchronous calls
if (!enc.compute_sync()) {
// if this happens it's a bug in the library.
// the **Decoder** can fail, but the **Encoder** can never fail.
std::cout << "Enc-RaptorQ failure! really bad!\n";
return false;
}
// the probability that a symbol will be dropped.
if (drop_probability > static_cast<float> (90.0))
drop_probability = 90.0; // this is still too high probably.
// we will store here all encoded and transmitted symbols
// std::pair<symbol id (esi), symbol data>
using symbol_id = uint32_t; // just a better name
std::vector<std::pair<symbol_id, std::vector<uint8_t>>> received;
{
// in this block we will generate the symbols that will be sent to
// the decoder.
// a block of size X will need at least X symbols to be decoded.
// we will randomly drop some symbols, but we will keep generating
// repari symbols until we have the required number of symbols.
std::uniform_real_distribution<float> drop_rnd (0.0, 100.0);
uint32_t received_tot = 0;
// Now get the source symbols.
// source symbols are specials because they contain the input data
// as-is, so if you get all of these, you don't need repair symbols
// to make sure that we are using the decoder, drop the first
// source symbol.
auto source_sym_it = enc.begin_source();
++source_sym_it; // ignore the first soure symbol (=> drop it)
source_sym_it++;
for (; source_sym_it != enc.end_source(); ++source_sym_it) {
// we save the symbol here:
// make sure the vector has enough space for the symbol:
// fill it with zeros for the size of the symbol
std::vector<uint8_t> source_sym_data (symbol_size, 0);
// save the data of the symbol into our vector
auto it = source_sym_data.begin();
auto written = (*source_sym_it) (it, source_sym_data.end());
if (written != symbol_size) {
// this can only happen if "source_sym_data" did not have
// enough space for a symbol (here: never)
std::cout << written << "-vs-" << symbol_size <<
" Could not get the whole source symbol!\n";
return false;
}
// can we keep this symbol or do we randomly drop it?
float dropped = drop_rnd (rnd);
if (dropped <= drop_probability) {
continue; // start the cycle again
}
// good, the symbol was received.
++received_tot;
// add it to the vector of received symbols
symbol_id tmp_id = (*source_sym_it).id();
received.emplace_back (tmp_id, std::move(source_sym_data));
}
std::cout << "Source Packet lost: " << enc.symbols() - received.size()
<< "\n";
//--------------------------------------------
// we finished working with the source symbols.
// now we need to transmit the repair symbols.
auto repair_sym_it = enc.begin_repair();
auto max_repair = enc.max_repair(); // RaptorQ can theoretically handle
// infinite repair symbols
// but computers are not so infinite
// we need to have at least enc.symbols() + overhead symbols.
for (; received.size() < (enc.symbols() + overhead) &&
repair_sym_it != enc.end_repair (max_repair);
++repair_sym_it) {
// we save the symbol here:
// make sure the vector has enough space for the symbol:
// fill it with zeros for the size of the symbol
std::vector<uint8_t> repair_sym_data (symbol_size, 0);
// save the data of the symbol into our vector
auto it = repair_sym_data.begin();
auto written = (*repair_sym_it) (it, repair_sym_data.end());
if (written != symbol_size) {
// this can only happen if "repair_sym_data" did not have
// enough space for a symbol (here: never)
std::cout << written << "-vs-" << symbol_size <<
" Could not get the whole repair symbol!\n";
return false;
}
// can we keep this symbol or do we randomly drop it?
float dropped = drop_rnd (rnd);
if (dropped <= drop_probability) {
continue; // start the cycle again
}
// good, the symbol was received.
++received_tot;
// add it to the vector of received symbols
symbol_id tmp_id = (*repair_sym_it).id();
received.emplace_back (tmp_id, std::move(repair_sym_data));
}
if (repair_sym_it == enc.end_repair (enc.max_repair())) {
// we dropped waaaay too many symbols!
// should never happen in real life. it means that we do not
// have enough repair symbols.
// at this point you can actually start to retransmit the
// repair symbols from enc.begin_repair(), but we don't care in
// this example
std::cout << "Maybe losing " << drop_probability << "% is too much?\n";
return false;
}
}
// Now we all the source and repair symbols are in "received".
// we will use those to start decoding:
// define "Decoder_type" to write less afterwards
using Decoder_type = RaptorQ::Decoder<
typename std::vector<uint8_t>::iterator,
typename std::vector<uint8_t>::iterator>;
Decoder_type dec (block, symbol_size, Decoder_type::Report::COMPLETE);
// "Decoder_type::Report::COMPLETE" means that the decoder will not
// give us any output until we have decoded all the data.
// there are modes to extract the data symbol by symbol in an ordered
// an unordered fashion, but let's keep this simple.
// we will store the output of the decoder here:
// note: the output need to have at least "mysize" bytes, and
// we fill it with zeros
std::vector<uint8_t> output (mysize, 0);
// now push every received symbol into the decoder
for (auto &rec_sym : received) {
// as a reminder:
// rec_sym.first = symbol_id (uint32_t)
// rec_sym.second = std::vector<uint8_t> symbol_data
symbol_id tmp_id = rec_sym.first;
auto it = rec_sym.second.begin();
auto err = dec.add_symbol (it, rec_sym.second.end(), tmp_id);
if (err != RaptorQ::Error::NONE && err != RaptorQ::Error::NOT_NEEDED) {
// When you add a symbol, you can get:
// NONE: no error
// NOT_NEEDED: libRaptorQ ignored it because everything is
// already decoded
// INITIALIZATION: wrong parameters to the decoder contructor
// WRONG_INPUT: not enough data on the symbol?
// some_other_error: errors in the library
std::cout << "error adding?\n";
return false;
}
}
// by now we now there will be no more input, so we tell this to the
// decoder. You can skip this call, but if the decoder does not have
// enough data it sill wait forever (or until you call .stop())
dec.end_of_input (RaptorQ::Fill_With_Zeros::NO);
// optional if you want partial decoding without using the repair
// symbols
// std::vector<bool> symbols_bitmask = dec.end_of_input (
// RaptorQ::Fill_With_Zeros::YES);
// decode, and do not return until the computation is finished.
auto res = dec.wait_sync();
if (res.error != RaptorQ::Error::NONE) {
std::cout << "Couldn't decode.\n";
return false;
}
// now save the decoded data in our output
size_t decode_from_byte = 0;
size_t skip_bytes_at_begining_of_output = 0;
auto out_it = output.begin();
auto decoded = dec.decode_bytes (out_it, output.end(), decode_from_byte,
skip_bytes_at_begining_of_output);
// "decode_from_byte" can be used to have only a part of the output.
// it can be used in advanced setups where you ask only a part
// of the block at a time.
// "skip_bytes_at_begining_of_output" is used when dealing with containers
// which size does not align with the output. For really advanced usage only
// Both should be zero for most setups.
if (decoded.written != mysize) {
if (decoded.written == 0) {
// we were really unlucky and the RQ algorithm needed
// more symbols!
std::cout << "Couldn't decode, RaptorQ Algorithm failure. "
"Can't Retry.\n";
} else {
// probably a library error
std::cout << "Partial Decoding? This should not have happened: " <<
decoded.written << " vs " << mysize << "\n";
}
return false;
} else {
std::cout << "Decoded: " << mysize << "\n";
}
// byte-wise check: did we actually decode everything the right way?
for (uint64_t i = 0; i < mysize; ++i) {
if (input[i] != output[i]) {
// this is a bug in the library, please report
std::cout << "The output does not correspond to the input!\n";
return false;
}
}
return true;
}
int main (void)
{
// get a random number generator
std::mt19937_64 rnd;
std::ifstream rand("/dev/urandom");
uint64_t seed = 0;
rand.read (reinterpret_cast<char *> (&seed), sizeof(seed));
rand.close ();
rnd.seed (seed);
// keep some computation in memory. If you use only one block size it
// will make things faster on bigger block size.
// allocate 5Mb
RaptorQ__v1::local_cache_size (5000000);
// for our test, we use an input of random size, between 100 and 10.000
// bytes.
std::uniform_int_distribution<uint32_t> distr(100, 10000);
uint32_t input_size = distr (rnd);
if (!test_rq (input_size, rnd, 20.0, 4))
return -1;
std::cout << "The example completed successfully\n";
return 0;
}
external/build_deps.cmake
View file @ a394b22a
...@@ -19,31 +19,68 @@ ...@@ -19,31 +19,68 @@
if(RQ_BUILD_LZ4) if(RQ_BUILD_LZ4)
# build lz4, then it will be statically linked in the library #
include(ExternalProject) # target: LZ4 static library
set(RQ_LZ4_BUILDNAME lz4_static) #
ExternalProject_Add(${RQ_LZ4_BUILDNAME} set(RQ_LZ4_VERSION_MAJOR 1)
GIT_REPOSITORY file://${PROJECT_SOURCE_DIR}/external/lz4 set(RQ_LZ4_VERSION_MINOR 5)
CONFIGURE_COMMAND "" set(RQ_LZ4_VERSION_PATCH r128)
BUILD_COMMAND make lib set(RQ_LZ4_VERSION_STRING " \"${RQ_LZ4_VERSION_MAJOR}.${RQ_LZ4_VERSION_MINOR}.${CPACK_PACKAGE_VERSION_PATCH}\" ")
BUILD_IN_SOURCE 1
INSTALL_COMMAND cp ${CMAKE_CURRENT_BINARY_DIR}/${RQ_LZ4_BUILDNAME}-prefix/src/${RQ_LZ4_BUILDNAME}/lib/liblz4.a ${CMAKE_CURRENT_BINARY_DIR}
)
add_custom_command( set(RQ_LZ4_LIBNAME rq_lz4_static)
OUTPUT lz4_deterministic.run
COMMAND make_deterministic ${CMAKE_CURRENT_BINARY_DIR}/liblz4.a set(RQ_LZ4_DIR ${PROJECT_SOURCE_DIR}/external/lz4/lib/)
DEPENDS ${RQ_LZ4_BUILDNAME} make_deterministic set(RQ_LZ4_SRCS_LIB ${RQ_LZ4_DIR}lz4.c
COMMENT "Removing creation date from lz4 library..." ${RQ_LZ4_DIR}lz4hc.c
VERBATIM ${RQ_LZ4_DIR}lz4.h
) ${RQ_LZ4_DIR}lz4hc.h
add_library(rq_lz4 STATIC IMPORTED) ${RQ_LZ4_DIR}lz4frame.c
set_target_properties(rq_lz4 PROPERTIES IMPORTED_LOCATION ${CMAKE_CURRENT_BINARY_DIR}/liblz4.a) ${RQ_LZ4_DIR}lz4frame.h
add_custom_target(LZ4 DEPENDS lz4_deterministic.run) ${RQ_LZ4_DIR}xxhash.c)
set(RQ_LZ4_DEP rq_lz4)
add_library(${RQ_LZ4_LIBNAME} STATIC ${RQ_LZ4_SRCS_LIB})
include(CheckCCompilerFlag)
check_c_compiler_flag("-fPIC" RQ_LZ4_PIC)
if(RQ_LZ4_PIC)
target_compile_options(
${RQ_LZ4_LIBNAME} PRIVATE
"-fPIC"
)
endif()
check_c_compiler_flag("-std=c99" RQ_FLAG_C_99)
if(RQ_FLAG_C_99)
target_compile_options(
${RQ_LZ4_LIBNAME} PRIVATE
"-std=c99"
)
endif()
set_target_properties(${RQ_LZ4_LIBNAME} PROPERTIES
SOVERSION "${RQ_LZ4_VERSION_MAJOR}.${RQ_LZ4_VERSION_MINOR}")
#set_property(TARGET lz4_static PROPERTY LIBRARY_OUTPUT_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/lib")
ADD_DEFINITIONS("-DLZ4_VERSION=\"${RQ_LZ4_VERSION_PATCH}\"")
INCLUDE_DIRECTORIES (SYSTEM PUBLIC ${RQ_LZ4_DIR})
# NOTE: will be used like "-Wl..flag,flag,flag" . no spaces, and keep the comma
set(RQ_LZ4_EXCLUDE_SYM ",--exclude-libs lib${RQ_LZ4_LIBNAME}.a")
# set the dependency and build it all
if(CMAKE_SYSTEM_NAME MATCHES "Windows")
add_custom_target(LZ4 DEPENDS ${RQ_LZ4_LIBNAME})
set(RQ_LZ4_DEP ${RQ_LZ4_LIBNAME})
else()
add_custom_command(
OUTPUT lz4_deterministic.run
COMMAND make_deterministic ${CMAKE_CURRENT_BINARY_DIR}/lib${RQ_LZ4_LIBNAME}.a
DEPENDS ${RQ_LZ4_LIBNAME} make_deterministic
COMMENT "Removing creation date from lz4 library..."
VERBATIM
)
add_custom_target(LZ4 DEPENDS lz4_deterministic.run)
set(RQ_LZ4_DEP ${RQ_LZ4_LIBNAME})
endif()
else() else()
add_custom_target(LZ4) add_custom_target(LZ4)
if (USE_LZ4 MATCHES "ON") if (USE_LZ4 MATCHES "ON")
set(RQ_LZ4_DEP lz4) set(RQ_LZ4_DEP ${RQ_LZ4_LIB})
endif() endif()
endif() endif()
external/eigen3/.hg_archival.txt
View file @ a394b22a
repo: 8a21fd850624c931e448cbcfb38168cb2717c790 repo: 8a21fd850624c931e448cbcfb38168cb2717c790
node: 07105f7124f9aef00a68c85e0fc606e65d3d6c15 node: 5a0156e40feb7c4136680b493c6e433d91a6f355
branch: 3.2 branch: 3.3
tag: 3.2.8 tag: 3.3.4
external/eigen3/.hgeol
View file @ a394b22a
[patterns] [patterns]
*.sh = LF
*.MINPACK = CRLF
scripts/*.in = LF scripts/*.in = LF
debug/msvc/*.dat = CRLF debug/msvc/*.dat = CRLF
debug/msvc/*.natvis = CRLF
unsupported/test/mpreal/*.* = CRLF unsupported/test/mpreal/*.* = CRLF
** = native ** = native
......