Distributed Data Loading¶
aeon-service is a component managing distributed data loading sessions. It does all heavy computing of data extraction, transformation and augmentation. Clients talk with service via REST-like API. Client may either create new sessions or connect to existing ones.
cpprestsdk library is used to implement REST service. It’s available on Ubuntu 16.04, but not on Ubuntu 14.04 or Centos. If that’s your problem, you need to build this library from sources. This might be helpful to achieve that. Version 2.4 is proved to be working on Centos 7.4.
On Ubuntu 16.04 and higher you need just:
sudo apt install libcpprest-dev
When using RDMA transfer, you need to install libfabric. Installation for Centos:
sudo yum install libfabric-devel
If it’s not available in your OS, you need to build it from sources:
git clone https://github.com/ofiwg/libfabric cd libfabric ./autogen.sh ./configure make -j sudo make install
The building of AEON client is enabled by default. To disable building AEON client please provide cmake flag
To enable building the AEON service (by default disabled) please provide cmake flag
By providing the flag the cmake will start resolving dependencies required for AEON service tragets.
The “must have” component is C++ REST SDK from Microsoft (both runtime and development files).
The packages are available in all recent Linux operating systems, for other supported systems the
cpprest package might not exist.
In such a case please build the library from source files. If you plan to install the library in non-standard directory, then let know
to the cmake its location by providing option
-DCPPREST_DIR (root directory of runtime and development files).
To enable RDMA support (by default disabled) please provide the cmake flag
-DENABLE_OPENFABRICS_CONNECTOR=ON. The flag is recognized
by AEON service and AEON client targets. The “must have” component in this case is OpenFrabics Interface library (runtime and development
files). The packages are available for all recent Linux operating systems, and for other supported systems please build the library from
source files. If you plan to install the library in non-standard directory, then let know to the cmake its location by providing the
-DOPENFABRICS_DIR (root directory of runtime and development files).
Currently only data parallelism is supported. That means that each client will get different batch in every iteration step. If there are
B batches in whole dataset,
C clients and they are synchronized to process batch after batch, then each client will get only
To initialize dataloading session, you need to create one. It can be done in two ways. The first is to send
POST request to
/api/v1/dataset with config in the body. In response you will get session id. The second one is to create aeon object as usual, but providing
remote object without
session_id field. In this case aeon will create new session, which can be passed to other clients. Session id can be fetched from aeon object field
Session is destroyed along with dataloader object assuming that
close_session is not set to
You can use separate dataloading sessions just by not using
session_id field in configuration.
To improve batch fetching speed, you may consider using RDMA transfer. For this, you need to install libfabric library.
On service side you need to provide
port parameters to enable RDMA transfer.
On client side, you have to add parameters
remote object to make use of RDMA transfer.
Currently RDMA works only on adapters supporting IB verbs, including Intel® Omni-Path Architecture adapters.
aeon-service is a separate executable binary. Below you can find it’s parameters.
|uri||u||string||Required||URI to listening interface in format ‘protocol://host:port/’.|
|address||a||string||“”||IP address of RDMA interface.|
|port||p||uint||0||Port number of RDMA interface.|
|daemon||d||flag||-||Run the process in background as a daemon process. By default the process runs in foreground.|
|log||l||string||/var/log/aeon-service.log||Path to log file.|
RESTful API can be found here.
Connection with service is configurable with
remote object in aeon config. Below you can find it’s fields. If
remote object is absent, regular local data loading will happen.
remote object from main aeon config:
|address||string||Required||Service address to connect to.|
|port||uint||Required||Service port to connect to.|
|session_id||string||“”||ID of shared session to connect to. If it’s not provided, new session will be created.|
|close_session||bool||true||If set to true, aeon will close session when aeon object is being destroyed.|
|async||bool||true||async set to true makes batch loading to be double-buffered. Please note that async mode can make client fetch one batch more than requested.|
|rdma_address||string||“”||IP address of RDMA interface.|
|rdma_port||uint||0||Port number of RDMA interface.|
|debug_output_directory||string||“”||Writes received images to the provided directory.|