Firstly, it is important to note the primary differences between a C3 connector and a C4 connector:
In short, C4 connectors need a bit more thought put into their ergonomics than a standard C3 connector.
For example, you might make a "BulkPushProducts" node which collects all its inputs over the course of a pipeline run then
sends everything to the remote system in one go, requiring tracking an internal buffer of products.
Contrast this with a C3 connector, which would simply provide CreateProduct(prod) and CreateProducts(List prods) methods and expect
the programmer to track their own list.
Overall, the C3 portion of the connector should simply be thought of as the backing library, with the C4 portion acting as
scripting bindings to expose it with a standardized interface and a some extra utilities (nodes with internal logic) that ease
potential pain-points in using the more "raw" nodes. Use your imagination and try to anticipate the kind of Quality of Life
nodes that may come in handy for working with that system.
Finally, although this is written with C3 connectors in mind it can be applied equally to other libraries, such as Woo.NET or ShopifySharp. The basic idea
is still taking a lower-level library and exposing it in a higher-level, re-usable manner.
CancellationToken token argument, C4 uses a global cancellation token stored at ScriptResolver.RunParams.Token.[InPort] and a [Param], to allow users to configure it in either manner.CommonNode or CommonScript and implement IScript and INode's interfaces manually.
Utils.GeneralUtils.OverrideWithValuesFrom to allow the user to specify which fields to override directly in params:public abstract class BaseOverrideWithValuesFrom : CommonNode, INode
{
public IReadOnlySet<string> Params => Parameters.Keys.Prepend("$Dynamic").ToHashSet();
protected Dictionary<string, string> Parameters { get; set; } = new();
public string? GetParam(string path) => Parameters[path];
public void SetParam(string path, string? value) => Parameters[path] = value
...
$ and the rest of the keys are assumed to just be access paths.FieldToField<TSrc, TDest> is written as exactly that: public class FieldToField<TSrc, TDest> : CommonNode, INode
where TSrc : class
where TDest : class
{
[InPort] public TSrc? Input { get; set; }
...
[OutPort] public TDest? Output { get; set; }
...
DataTypes exposed by the C3 connector should be exposed to C4's TypeResolver for use in the mappings
editor.
Typically, this is done by adding [Record(Name = "AcmeTYPENAME")] attributes to each module,
where Acme is the name of the remote system and TYPENAME is the type's direct name.
You can use the type resolver hook to automatically expose types without needing to manually add the [Record]:
This doubles as an example of using
Converter.Hooks.Addto expose standard conversions for your module
namespace C4.Modules.GP2018
{
public class Config : TypeResolverConfig
{
public override string ERPName => "DynGP16";
public override async Task ResolveHook()
{
var allDataModelTypes = typeof(BusinessObject)
.GetCustomAttributes<KnownTypeAttribute>()
.Select(kt => kt.Type)
.Where(t => !type.Name.EndsWith("Base"));
foreach (var type in allDataModelTypes)
TypeResolver.RegisterType(ERPName, type, "GP2018" + type.Name);
Converter.Hooks.AddAll(new ConverterFunc[]{
async (from, value, into) => value is MoneyAmount amt && into == typeof(double)
? ConversionResult.Can((double)amt.Value)
: ConversionResult.Cant,
async (from, value, into) => value is MoneyAmount amt && into == typeof(double?)
? ConversionResult.Can((double?)amt.Value)
: ConversionResult.Cant,
async (from, value, into) => value is Quantity amt && into == typeof(double)
? ConversionResult.Can((double)amt.Value)
: ConversionResult.Cant,
...
Don't be afraid to add useful utility properties to types. For example, if an AcmeProduct has a
List<{string Field; string Value}> CustomFields property on it, but there exists a well-known custom field
such as ExternalID, you might implement a property like
public string ExternalID {
get => CustomFields.SingleOrDefault(f => f.Field == "ExternalID").Value;
set => (CustomFields.SingleOrDefault(f => f.Field == "ExternalID") ?? CustomFields.Add(new() { Field = "ExternalID" })).Value = value; // Pseudocode-ish
}
to simplify mapping to that property. A more complicated example could be to implement a property with an
index-operator overload to allow exposing custom fields like a Dictionary<string, string> rather than the
List<{Key; Value}> Acme's API expects it to be transmitted. As always, make it ergonomic for the mapper
and remember that we don't have LINQ in the mapping interface (yet).
A common pattern in C4 connectors is to make a Syncer base class for your nodes that contains all needed parameters:
public abstract class AcmeSyncer : /* Gives us [Param] and [*Port] */ CommonNode, IScript {
[Param] public string BaseURL { get; set; }
[Param] public string User { get; set; }
[Param] public string Pass { get; set; }
// The C3 integration service
public AcmeIntegrationService Service { get; set; }
async Task IScript.SettleParams() => this.SettleParams();
// Virtual so individual nodes can override it to add their own param checks.
protected async virtual Task SettleParams() {
// Check that BaseURL/User/Pass are all good (sometimes including a test request)
// Set up Service with all needed details
}
}
You could build further base classes for common node kinds like so
// Find all of a given type
public abstract class AcmePicker<T> : AcmeSyncer, INode
{
[InPort, Param] public DateTime? UpdatedSince { get; set; }
// ...other inports/params
// Each indivudal record from Acme, updated each Pump.
[OutPort] public T? Output => _items.Current;
protected IAsyncEnumerable<T> _items;
[CtrlPort] public Flow Ctrl { get; set; }
async Task INode.OnPipelineStart() => _items = Pick();
async Task INode.Pump()
{
// Would typically want to provide a way of configuring which action instead of hardcoding Halt.
Ctrl = (await _items.MoveNextAsync()) ? Flow.Continue : Flow.Halt;
}
protected abstract IAsyncEnumerable<T> Pick();
}
these extra base classes make the individual bindings much shorter and easier to maintain.
We won't use these base classes in the examples below in order to show the more "raw" way of doing things.
The first step is typically creating direct bindings for the most basic CRUD operations the C3 connector exposes. For example, the methods
// In AcmeIntegrationService
async Task<Product> GetProductByID(int id, CancellationToken token) { ... }
async Task<Product> GetProductBySKU(string sku, CancellationToken token) { ... }
could be translated into nodes like so
public class PickAcmeProducts : AcmePicker<Product>, INode {
[InPort] public object? Key { get; set; }
[OutPort] public Product? Product { get; set; }
async Task INode.Pump() => Product = Key switch {
null => null,
int id => await Service.GetProductByID(id),
string sku => await Service.GetProductBySKU(sku),
// etc
};
}
this actually wraps both methods in a single node and uses typechecking to figure out what the user intended,
making things easier to work with for the end user.
Even if you can't make the implementation generic, try to expose the pipeline/user-facing interface generically.
For example, the GP module is implemented with much copy and paste, but exposes everything to the resolver
through standardized generic interfaces:
[Script(Name = "GP2018Pusher<GP2018SalesItem>")]
public class GP2018ItemPusher : GP2018Pusher<SalesItem>
So although it's technically a non-generic class, the ScriptResolver will see and resolve it as if it were generic. This also means it will work with aliases (if GP2018SalesItem were aliased to GP2018Product, you
could do GP2018Pusher<GP2018Product> in a pipeline and it would still work).
Sometimes a system may need some more complex logic that's not as easy to represent in the graph. Some examples:
As always, you don't need to follow this exactly or at all, just use your imagination to find the way that'll make
things the most ergonomic both internally in the code and externally in the pipelines; ergonomics and maintainability are all that matter here.