Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/concepts/application-building-blocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Instead of building up a static dataflow DAG, these functions can communicate wi
If you are familiar with actor programming, this does share certain similarities in its ability to dynamically message between components.
However, there are a number of significant differences.

## Persisted States
### Persisted States

The first is that all functions have locally embedded state, known as persisted states.

Expand All @@ -66,9 +66,9 @@ The first is that all functions have locally embedded state, known as persisted
One of Apache Flink's core strengths is its ability to provide fault-tolerant local state.
When inside a function, while it is performing some computation, you are always working with local state in local variables.

## Fault Tolerance
### Fault Tolerance

For both state and messaging, Stateful Function's is still able to provide the exactly-once guarantees users expect from a modern data processessing framework.
For both state and messaging, Stateful Functions is able to provide the exactly-once guarantees users expect from a modern data processessing framework.

<p class="text-center">
<img width="80%" src="{{ site.baseurl }}/fig/concepts/statefun-app-fault-tolerance.svg"/>
Expand All @@ -78,9 +78,9 @@ In the case of failure, the entire state of the world (both persisted states and

These guarantees are provided with no database required, instead Stateful Function's leverages Apache Flink's proven snapshotting mechanism.

## Event Egress
### Event Egress

Finally, applications can output data to external systems via event egress's.
Finally, applications can output data to external systems via event egresses.

<p class="text-center">
<img width="80%" src="{{ site.baseurl }}/fig/concepts/statefun-app-egress.svg"/>
Expand Down
4 changes: 2 additions & 2 deletions docs/concepts/distributed_architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ In addition to the Apache Flink processes, a full deployment requires [ZooKeeper

## Logical Co-location, Physical Separation

A core principle of many Stream Processors is that application logic and the application state must be co-located. That approach is the basis for their out-of-the box consistency. Stateful Function takes a unique approach to that by *logically co-locating* state and compute, but allowing to *physically separate* them.
A core principle of many Stream Processors is that application logic and the application state must be co-located. That approach is the basis for their out-of-the box consistency. Stateful Functions takes a unique approach to that by *logically co-locating* state and compute, but allowing to *physically separate* them.

- *Logical co-location:* Messaging, state access/updates and function invocations are managed tightly together, in the same way as in Flink's DataStream API. State is sharded by key, and messages are routed to the state by key. There is a single writer per key at a time, also scheduling the function invocations.

Expand All @@ -67,7 +67,7 @@ The stateful functions themselves can be deployed in various ways that trade off

*Remote Functions* use the above-mentioned principle of *physical separation* while maintaining *logical co-location*. The state/messaging tier (i.e., the Flink processes) and the function tier are deployed, managed, and scaled independently.

Function invocations happen through an HTTP / gRPC protocol and go through a service that routes invocation requests to any available endpoint, for example a Kubernetes (load-balancing) service, the AWS request gateway for Lambda, etc. Because invocations are self-contained (contain message, state, access to timers, etc.) the target functions can treated like any stateless application.
Function invocations happen through an HTTP / gRPC protocol and go through a service that routes invocation requests to any available endpoint, for example a Kubernetes (load-balancing) service, the AWS request gateway for Lambda, etc. Because invocations are self-contained (contain message, state, access to timers, etc.) the target functions can be treated like any stateless application.

<p class="text-center">
<img width="80%" src="{{ site.baseurl }}/fig/concepts/arch_funs_remote.svg"/>
Expand Down
10 changes: 5 additions & 5 deletions docs/concepts/logical.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,23 +34,23 @@ Users are encouraged to model their applications as granularly as possible, base
## Function Address

In a local environment, the address of an object is the same as a reference to it.
But in a Stateful Function's application, function instances are virtual and their runtime location is not exposed to the user.
Instead, an ``Address`` is used to reference a specific stateful function in the system..
But in a Stateful Functions application, function instances are virtual and their runtime location is not exposed to the user.
Instead, an ``Address`` is used to reference a specific stateful function in the system.

<p class="text-center">
<img width="80%" src="{{ site.baseurl }}/fig/concepts/address.svg"/>
</p>

An address is made of two components, a ``FunctionType`` and ``ID``.
A function type is similar to a class in an object-oriented language; it declares what sort of function the address references.
The id is a primary key, which scopes the function call to a specific instance of the function type.
The ID is a primary key, which scopes the function call to a specific instance of the function type.

When a function is being invoked, all actions - including reads and writes of persisted states - are scoped to the current address.

For example, imagine there was a Stateful Function application to track the inventory of a warehouse.
For example, imagine there was a Stateful Functions application to track the inventory of a warehouse.
One possible implementation could include an ``Inventory`` function that tracks the number units in stock for a particular item; this would be the function type.
There would then be one logical instance of this type for each SKU the warehouse manages.
If it were clothing, there might be an instance for shirts and another for pants; "shirt" and "pant" would be two ids.
If it were clothing, there might be an instance for shirts and another for pants; "shirt" and "pant" would be two IDs.
Each instance may be interacted with and messaged independently.
The application is free to create as many instances as there are types of items in inventory.

Expand Down
2 changes: 1 addition & 1 deletion docs/deployment-and-operations/packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ COPY module.yaml /opt/statefun/modules/remote/module.yaml
{% if site.is_stable %}
<div class="alert alert-info">
The Flink community is currently waiting for the official Docker images to be published to Docker Hub.
In the meantime, Ververica has volunteered to make Stateful Function's images available via their public registry:
In the meantime, Ververica has volunteered to make Stateful Functions' images available via their public registry:

<code class="language-dockerfile" data-lang="dockerfile">
<span class="k">FROM</span><span class="s"> ververica/flink-statefun:{{ site.version }}</span>
Expand Down
7 changes: 5 additions & 2 deletions docs/getting-started/java_walkthrough.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,18 +93,19 @@ private static String greetText(String name, int seen) {
return String.format("Happy to see you once again %s !", name);
default:
return String.format("Hello at the %d-th time %s", seen + 1, name);
}
}
{% endhighlight %}

## Routing Messages

To send a user a personalized greeting, the system needs to keep track of how many times it has seen each user so far.
To send a personalized greeting to a user, the system needs to keep track of how many times it has seen each user so far.
Speaking in general terms, the simplest solution would be to create one function for every user and independently track the number of times they have been seen. Using most frameworks, this would be prohibitively expensive.
However, stateful functions are virtual and do not consume any CPU or memory when not actively being invoked.
That means your application can create as many functions as necessary — in this case, users — without worrying about resource consumption.

Whenever data is consumed from an external system (or [ingress]({{ site.baseurl }}/io-module/index.html#ingress)), it is routed to a specific function based on a given function type and identifier.
The function type represents the Class of function to be invoked, such as the Greeter function, while the identifier (``GreetRequest#getWho``) scopes the call to a specific virtual instance based on some key.
The function type represents the class of function to be invoked, such as the Greeter function, while the identifier (``GreetRequest#getWho``) scopes the call to a specific virtual instance based on some key.

{% highlight java %}
package org.apache.flink.statefun.examples.greeter;
Expand Down Expand Up @@ -200,3 +201,5 @@ docker-compose exec kafka-broker kafka-console-consumer.sh \

This Greeter never forgets a user.
Try and modify the function so that it will reset the ``count`` for any user that spends more than 60 seconds without interacting with the system.

Check out the [Java SDK]({{ site.baseurl }}/sdk/java.html) page for more information on how to achieve this.
2 changes: 1 addition & 1 deletion docs/getting-started/project-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ $ tree statefun-quickstart/
The project contains four files:

* ``pom.xml``: A pom file with the basic dependencies to start building a Stateful Functions application.
* ``Module``: The entry point for the application.
* ``Module.java``: The entry point for the application.
* ``org.apache.flink.statefun.sdk.spi.StatefulFunctionModule``: A service entry for the runtime to find the module.
* ``Dockerfile``: A Dockerfile to quickly build a Stateful Functions image ready to deploy.

Expand Down
18 changes: 10 additions & 8 deletions docs/getting-started/python_walkthrough.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,16 +170,16 @@ from statefun import StatefulFunctions
functions = StatefulFunctions()

@functions.bind("example/greeter")
def greet(context, message: GreetRequest):
def greet(context, greet_request: GreetRequest):
response = GreetResponse()
response.name = message.name
response.greeting = "Hello {}".format(message.name)
response.name = greet_request.name
response.greeting = "Hello {}".format(greet_request.name)

egress_message = kafka_egress_record(topic="greetings", key=message.name, value=response)
egress_message = kafka_egress_record(topic="greetings", key=greet_request.name, value=response)
context.pack_and_send_egress("example/greets", egress_message)
{% endhighlight %}

For each message, a response is constructed and sent to a kafka topic call `greetings` partitioned by `name`.
For each message, a response is constructed and sent to a Kafka topic called `greetings` partitioned by `name`.
The `egress_message` is sent to a an `egress` named `example/greets`.
This identifier points to a particular Kafka cluster and is configured on deployment below.

Expand Down Expand Up @@ -211,7 +211,7 @@ For each user, functions can now track how many times they have been seen.

{% highlight python %}
@functions.bind("example/greeter")
def greet(context, greet_message: GreetRequest):
def greet(context, greet_request: GreetRequest):
state = context.state('seen_count').unpack(SeenCount)
if not state:
state = SeenCount()
Expand All @@ -226,7 +226,7 @@ def greet(context, greet_message: GreetRequest):
context.pack_and_send_egress("example/greets", egress_message)
{% endhighlight %}

The state, `seen_count` is always scoped to the current name so it can track each user independently.
The state `seen_count` is always scoped to the current name so it can track each user independently.

## Wiring It All Together

Expand All @@ -243,7 +243,7 @@ from statefun import RequestReplyHandler

functions = StatefulFunctions()

@functions.bind("walkthrough/greeter")
@functions.bind("example/greeter")
def greeter(context, message: GreetRequest):
pass

Expand Down Expand Up @@ -362,6 +362,8 @@ docker-compose logs -f event-generator
This Greeter never forgets a user.
Try and modify the function so that it will reset the ``seen_count`` for any user that spends more than 60 seconds without interacting with the system.

Check out the [Python SDK]({{ site.baseurl }}/sdk/python.html) page for more information on how to achieve this.

## Full Application

{% highlight python %}
Expand Down
36 changes: 18 additions & 18 deletions docs/sdk/java.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ under the License.
Stateful functions are the building blocks of applications; they are atomic units of isolation, distribution, and persistence.
As objects, they encapsulate the state of a single entity (e.g., a specific user, device, or session) and encode its behavior.
Stateful functions can interact with each other, and external systems, through message passing.
The Java SDK is supported as an [embedded_module]({{ site.baseurl }}/sdk/modules.html#embedded-module).
The Java SDK is supported as an [embedded module]({{ site.baseurl }}/sdk/modules.html#embedded-module).

To get started, add the Java SDK as a dependency to your application.

Expand Down Expand Up @@ -96,16 +96,16 @@ public class FnMatchGreeter extends StatefulMatchFunction {
.predicate(Employee.class, this::greetEmployee);
}

private void greetManager(Context context, Employee message) {
System.out.println("Hello manager " + message.getEmployeeId());
private void greetCustomer(Context context, Customer message) {
System.out.println("Hello customer " + message.getName());
}

private void greetEmployee(Context context, Employee message) {
System.out.println("Hello employee " + message.getEmployeeId());
}

private void greetCustomer(Context context, Customer message) {
System.out.println("Hello customer " + message.getName());
private void greetManager(Context context, Employee message) {
System.out.println("Hello manager " + message.getEmployeeId());
}
}
{% endhighlight %}
Expand Down Expand Up @@ -134,20 +134,20 @@ public class FnMatchGreeterWithCatchAll extends StatefulMatchFunction {
.otherwise(this::catchAll);
}

private void catchAll(Context context, Object message) {
System.out.println("Hello unexpected message");
private void greetCustomer(Context context, Customer message) {
System.out.println("Hello customer " + message.getName());
}

private void greetManager(Context context, Employee message) {
System.out.println("Hello manager");
private void greetEmployee(Context context, Employee message) {
System.out.println("Hello employee " + message.getEmployeeId());
}

private void greetEmployee(Context context, Employee message) {
System.out.println("Hello employee");
private void greetManager(Context context, Employee message) {
System.out.println("Hello manager " + message.getEmployeeId());
}

private void greetCustomer(Context context, Customer message) {
System.out.println("Hello customer");
private void catchAll(Context context, Object message) {
System.out.println("Hello unexpected message");
}
}
{% endhighlight %}
Expand All @@ -162,7 +162,7 @@ Finally, if a catch-all exists, it will be executed or an ``IllegalStateExceptio

## Function Types and Messaging

In Java, function types are defined as a _stringly_ typed reference containing a namespace and name.
In Java, function types are defined as logical pointers composed of a namespace and name.
The type is bound to the implementing class in the [module]({{ site.baseurl }}/sdk/modules.html#embedded-module) definition.
Below is an example function type for the hello world function.

Expand Down Expand Up @@ -228,7 +228,7 @@ public class FnDelayedMessage implements StatefulFunction {
## Completing Async Requests

When interacting with external systems, such as a database or API, one needs to take care that communication delay with the external system does not dominate the application’s total work.
Stateful Functions allows registering a java ``CompletableFuture`` that will resolve to a value at some point in the future.
Stateful Functions allows registering a Java ``CompletableFuture`` that will resolve to a value at some point in the future.
Future's are registered along with a metadata object that provides additional context about the caller.

When the future completes, either successfully or exceptionally, the caller function type and id will be invoked with a ``AsyncOperationResult``.
Expand Down Expand Up @@ -303,7 +303,7 @@ The data is always scoped to a specific function type and identifier.
Below is a stateful function that greets users based on the number of times they have been seen.

<div class="alert alert-info">
<strong>Attention:</strong> All <b>PersistedValue</b>, <b>PersistedTable</b>, and <b>PersistedAppendingBuffer</b> fields must be marked with an <b>@Persisted</b> annotation or they will not be made fault tolerant by the runtime.
<strong>Attention:</strong> All <b>PersistedValue</b>, <b>PersistedTable</b>, and <b>PersistedAppendingBuffer</b> fields must be marked with a <b>@Persisted</b> annotation or they will not be made fault tolerant by the runtime.
</div>

{% highlight java %}
Expand Down Expand Up @@ -345,7 +345,7 @@ public class FnUserGreeter implements StatefulFunction {
}
{% endhighlight %}

Persisted value comes with the right primitive methods to build powerful stateful applications.
``PersistedValue`` comes with the right primitive methods to build powerful stateful applications.
Calling ``PersistedValue#get`` will return the current value of an object stored in state, or ``null`` if nothing is set.
Conversely, ``PersistedValue#set`` will update the value in state and ``PersistedValue#clear`` will delete the value from state.

Expand Down Expand Up @@ -403,7 +403,7 @@ State TTL configurations are made fault-tolerant by the runtime. In the case of
## Function Providers and Dependency Injection

Stateful functions are created across a distributed cluster of nodes.
``StatefulFunctionProvider`` is a factory class for creating a new instance of a stateful function the first time it is activated.
``StatefulFunctionProvider`` is a factory class for creating a new instance of a ``StatefulFunction`` the first time it is activated.

{% highlight java %}
package org.apache.flink.statefun.docs;
Expand Down
4 changes: 2 additions & 2 deletions docs/sdk/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ specific language governing permissions and limitations
under the License.
-->

Stateful Function applications are composed of one or more ``Modules``.
Stateful Functions applications are composed of one or more modules.
A module is a bundle of functions that are loaded by the runtime and available to be messaged.
Functions from all loaded modules are multiplexed and free to message each other arbitrarily.

Expand Down Expand Up @@ -72,7 +72,7 @@ org.apache.flink.statefun.docs.BasicFunctionModule

Remote modules are run as external processes from the Apache Flink® runtime; in the same container, as a sidecar, or other external location.

This module type can support any number of language SDK's.
This module type can support any number of language SDKs.
Remote modules are registered with the system via ``YAML`` configuration files.

### Specification
Expand Down