Introduction to Coral

This page introduces you to Coral and its Application Programming Interface (API). If you’re new to Coral concepts, continue reading! If you just want to dive in and see Coral API in action first, select a language and try its Quick start.

Overview

In Coral, a client application can directly call an accelerator on a local hardware resource as if it were a common software function, making it easier for you to accelerate distributed applications and services. As in many RPC systems, Coral is based around the idea of defining a platform, specifying the accelerators that can be called remotely with their arguments and configuration parameters. On the server side, Coral dynamically implements an interface and runs a server to handle and respond to client calls. On the client side, the application just requests the accelerators available at the server. Coral relies on shared memory namespaces and uses gRPC as its underlying message interchange system.

Coral clients and servers can run and talk to each other in a variety of environments - from bare-metal Linux servers to containers inside a Kubernetes cluster - and applications can be written in any of Coral API’s supported languages. So, for example, you can easily use the same accelerator with a Coral client in C++ or Python.

Core concepts, architecture and lifecycle

Coral is a fast and general-purpose accelerator orchestrator. It is shipped with high-level APIs in C/C++, Java, and Python, and a unified engine that supports every heterogeneous multi-accelerator platform.

Using the API

On the client side, the application instantiates local objects known as requests that correspond to accelerators available at the server. The application can then just populate those requests with data in familiar data types and structures and submit them - Coral API looks after wrapping the arguments for the accelerator in the appropriate format, sending the request(s) to the server and returning the server’s response(s).

In many scenarios it’s useful to be able to start RPCs without blocking the current thread. The Coral programming API in all languages comes in an asynchronous flavor. You can find out more in each language’s tutorial.

RPC life cycle

In this section, you’ll take a closer look at what happens when a Coral client requests an accelerator. Let’s consider the simplest type of RPC where the client sends a single request and gets back a single response.

1. Once the client submits an accelerator request, Coral is notified that the RPC has been invoked with the client’s metadata for this request, the accelerator name, and the specified arguments.

2. Once the server has the client’s request, it does whatever work is necessary to run the accelerator and populate a response. The response is then returned to the client together with status details (status code) and optional trailing messages.

3. Then the client gets the response and checks if the response status is OK, which completes the call on the client side.

Quick start

Get InAccel Coral API

apt install coral-api

yum install coral-api

apt install coral-api

yum install coral-api

groupId: com.inaccel
artifactId: coral-api
version: X.Y.Z

pip install coral-api

Example program

#include <inaccel/coral.h>

#include <inaccel/coral>

import com.inaccel.coral.*;

import io.netty.buffer.ByteBuf;

import java.util.concurrent.Future;

import inaccel.coral as inaccel
import numpy as np

Supported argument types

An argument can be a scalar data type (single value; such as an int, a float, or a user-defined structure), or an array data type (one-dimensional collection of values).

Scalar arguments

	int size = 1024;

	int size = 1024;

		int size = 1024;

size = np.int32(1024)

For example, a 32-bit integer value.

Array arguments

Shared memory objects stored as a block of contiguous memory and used to hold data in a Coral program. Coral API maintains memory consistency in a coarse-grained fashion in regions of arrays. Memory consistency is guaranteed at synchronization points.

To allocate a zero-initialized array that can be shared by the host and all accelerators, use the language-native InAccel allocator:

allocate

	float *a = (float *) inaccel_alloc(size * sizeof(float));
	float *b = (float *) inaccel_alloc(size * sizeof(float));
	float *c = (float *) inaccel_alloc(size * sizeof(float));

	inaccel::vector<float> a(size), b(size), c(size);

		ByteBuf a = InAccelByteBufAllocator.DEFAULT.buffer(size * Float.BYTES);
		ByteBuf b = InAccelByteBufAllocator.DEFAULT.buffer(size * Float.BYTES);
		ByteBuf c = InAccelByteBufAllocator.DEFAULT.buffer(size * Float.BYTES);

with inaccel.allocator:
    a = np.ndarray(size, dtype=np.float32)
    b = np.ndarray(size, dtype=np.float32)
    c = np.ndarray(size, dtype=np.float32)

Defining the request

	inaccel_request request = inaccel_request_create("vector.addition");

	inaccel::request request("vector.addition");

		InAccel.Request request = new InAccel.Request("vector.addition");

request = inaccel.request('vector.addition')

Our next step is to instantiate the accelerator request and define the argument list. To define a request, you specify the name of the target accelerator.

For this example, consider the following accelerator:

void vadd(const float *a, const float *b, float *c, int size) { ... }

To submit a request, the accelerator arguments must be set. To set the argument value for a specific argument of an accelerator, call:

	inaccel_request_arg_array(request, size * sizeof(float), a, 0);
	inaccel_request_arg_array(request, size * sizeof(float), b, 1);
	inaccel_request_arg_array(request, size * sizeof(float), c, 2);
	inaccel_request_arg_scalar(request, sizeof(int), &size, 3);

	request.arg(a).arg(b).arg(c).arg(size);

		request.arg(a).arg(b).arg(c).arg(size);

request.arg(a).arg(b).arg(c).arg(size)

If the accelerator argument is declared to be a pointer of a built-in, or a user-defined type, the specified argument value must be an appropriate array region. For all other accelerator arguments (scalars), it must be a reference to the actual data.

Submit the request

To enqueue a request to execute an accelerator, call the submit function:

	inaccel_response response = inaccel_response_create();

	inaccel_submit(request, response);

	std::future<void> response = inaccel::submit(request);

		Future<Void> response = InAccel.submit(request);

response = inaccel.submit(request)

The response object is used to track the status of a request.

Wait for the response

	inaccel_response_wait(response);

	response.get();

		response.get();

response.result()

This function waits on the host thread for requests identified by response objects to complete. The responses act as synchronization points.

	inaccel_response_release(response);

	inaccel_request_release(request);

	inaccel_free(c);
	inaccel_free(b);
	inaccel_free(a);

		c.release();
		b.release();
		a.release();

Try it out!

gcc src/test/c/example.c -lcoral-api

./a.out

g++ src/test/cpp/example.cpp -lcoral-api -pthread

./a.out

gradle example

python src/test/python/example.py

Congratulations! You’ve just run an accelerated application with Coral.