In this example, we'll be integrating Magento and Dynamics GP, with an emphasis on the Magento
side.
When making any modules for C4, use the following namespacing convention:
C4.Modules.FOO For any ESM pertaining to system FOO
GP2018 vs DynamicsGP vs ...), so ifDynamicsGreatPlains2018 is verboseDynamicsGP2018 is fineGP is too short - which version of GP?DynamicsGP2018 and GP2018 are both fine. They specify the system and version easily enough.Magento is fine. They seem to keep a relatively stable API, so the version is less important. (for now)C4.Modules.CSM.ClientName for any CSM pertaining to ClientName
This is the easiest bit. Generate (automatically or by hand) any classes that'll be needed by
either the pipeline logic or any of your nodes. You may need to revisit this step later, if you
realize you need more.
To start, write a Config.cs, to tell the type resolver which System/ERP it should place any [Record]
types under by default:
public class Config : TypeResolverConfig
{
public override string ERPName => "Magento";
}
Any types that should be exposed to the type resolver should either be annotated
with [Record] or registered using the ResolveHook from the config class. An example for GP:
public class Config : TypeResolverConfig
{
public override string ERPName => "GP2018";
public override async Task ResolveHook()
{
foreach (var type in typeof(BusinessObject).GetCustomAttributes<KnownTypeAttribute>().Select(kt => kt.Type))
{
if (type.Name.EndsWith("Base")) { continue; }
TypeResolver.RegisterType(ERPName, type, "GP2018" + type.Name);
}
}
}
}
The above config scans GP's DLL and registers any non-base types under the naming convention GP2018XXXX.
Additionally, add any useful properties/attributes for data access or networking. For example, I annotate my Magento
classes with their endpoints:
[Record(Name = "MagentoCustomer", Categories = new[] { "Customers" })]
[EndpointInfo(
// Get all
ListFull = "/rest/V1/customers/search",
// Get by SKU
Read = "/rest/V1/customers/{{id}}",
Create = "/rest/V1/customers",
Update = "/rest/V1/customers/{{id}}")]
public partial class MagentoCustomer : MagentoBase
{
[JsonIgnore] // Used in networking
public override string? SaveWrapperField => "customer";
[JsonIgnore] // Used in networking, in case it's a different name than Id
public override int? ID => Id;
// Could be a pain to map to in the FieldToField editor but is commmonly used, so I make a property to get/set it
[JsonIgnore]
public string? TargetCustomerID
{
get => ...
set => ...
}
}
Note: The
[EndpointInfo]attribute is in core C4, but neither its precise meaning nor the available
{{templates}}are standardized.
PUT /crud/RemoteSystem
[
{ "Name": "Magento" },
{ "Name": "GP2018" }
]
Note: This will eventually be doable from the UI
Start C4, go to /mappings/from/GP2018/to/Magento, and map away. See Understanding Mappings for more.
Note: The runset philosophy of "sync everything into R0 and operate on that" isn't quite ready for prime-time
due to R0's inability to handle multiple primary keys (for now), so below I focus more on using them as nodes.
Write a picker and pusher for every entity in the system you'll need to save or load. Shortened example for Magento:
public abstract class MagentoSyncer<TMe, TEnt> : CommonNode<TMe>, IScript
where TEnt : MagentoBase
{
[Param]
public string BaseURL { get => baseURL; set => baseURL = value.TrimEnd('/'); }
[Param]
public string Token { get; set; }
async Task<string?> IScript.SettleParams() { ... }
}
public class MagentoPicker<TEnt> : MagentoSyncer<MagentoPicker<TEnt>, TEnt>, IPick<TEnt>, INode
where TEnt : MagentoBase
{
public async IAsyncEnumerable<TEnt> FindAll(PickHints hints) { /* REST code for a list */ }
/// <summary>
/// Pick an entity from Magento based on ID (for stuff like orders/customers) or a CSV of parent-child keys.
/// </summary>
public async Task<TEnt?> PickOne(string key) { ... }
[CtrlPort]
public Flow Ctrl { get; set; }
[OutPort]
public TEnt? Output { get; set; }
IAsyncEnumerator<TEnt> Items { get; set; }
async Task INode.OnPipelineStart() => Items = FindAll(new() { UpdatedSince = ScriptResolver.RunParams.LastRun }).GetAsyncEnumerator();
async Task INode.Pump()
{
try
{
Ctrl = await Items.MoveNextAsync() ? Flow.Continue : Flow.Halt;
Output = Items.Current;
}
catch (Exception e)
{
Ctrl = Flow.Err(e.Message);
Output = null;
}
}
}
public class MagentoPusher<TEnt> : MagentoSyncer<MagentoPusher<TEnt>, TEnt>, IPush<TEnt>, INode
where TEnt : MagentoBase
{
public async Task<string?> PushEnt(TEnt ent) { ... }
[CtrlPort]
public Flow Ctrl { get; set; }
[InPort]
public TEnt? Input { get; set; }
[OutPort]
public int? CreatedID { get; set; }
async Task INode.Pump()
{
Ctrl = Input is null ? Flow.Halt : Flow.Continue;
CreatedID = null;
if (Input is null) { return; }
CreatedID = int.TryParse(await PushEnt(Input), out var i) ? i : null;
Input = default;
Ctrl = CreatedID is null ? Flow.Err("Failed to create") : Flow.Continue;
}
}
Also create any helper nodes you think you'll need. For example, for resolving a Magento customer by Email:
public class MagentoReadCustomerByEmail : MagentoPicker<MagentoCustomer>, INode
{
[InPort]
public string? Email { get; set; }
[OutPort]
public MagentoCustomer? Customer { get; set; }
[CtrlPort]
public Flow Ctrl { get; set; }
async Task INode.Pump()
{
Ctrl = Flow.Continue;
if (Email.NullIfWhitespace() is null)
{
Customer = null;
Ctrl = Flow.SkipCycle;
}
else
{
Customer = await FindAll(new() { Filters = new() { ["email"] = new(Arena.FilterOp.Eq, Email) } }).SingleOrDefaultAsync();
}
}
}
}
In short, each ESM should contain:
For example: in the case of Western Truck, we need to hit the client's Magento DB directly in order to find a
customer by their
GP ID. For this, I took the existing EFCore DBContext, put it in a new WesternDB project in the types folder, and
made a WesternCSM project with one script:
public class FindMagentoCustomerIDByTargetCustomerID : CommonNode<FindMagentoCustomerIDByTargetCustomerID>, INode
{
[Param]
public string ConnStr { get; set; }
[InPort]
public object Input { get; set; }
[OutPort]
public int? CustomerID { get; set; }
[CtrlPort]
public Flow Ctrl { get; set; }
public async Task<int> FindIDByKey(string key)
{
using var db = new WesternDBContext(ConnStr);
return (int)await db
.RancocustomerGridFlats.AsQueryable()
.Where(rc => rc.TargetCustomerId == key)
.Select(rc => rc.EntityId)
.SingleAsync();
}
async Task INode.Pump()
{
Ctrl = Flow.Continue;
CustomerID = Input switch
{
Customer cust => await FindIDByKey(cust.Key.Id),
string key => await FindIDByKey(key),
_ => null,
};
if (CustomerID is null) { Ctrl = Flow.SkipCycle; }
}
}
These sorts of scripts actually could've normally been refactored into a MagentoGPESM. However, in our case,
the DB is very specific to the client (the Ranco prefixes on everything), so it makes more sense to make it a CSM.
Q: Do I put my stuff in the CSM or the ESM?
If it's anything that is unlikely to be useful for another client,
it belongs in the CSM. For example, CSMs should contain any EFCore DB contexts
and their associated nodes. In the case that something is somewhat likely to be useful for other clients,
but you're not sure, just bring it up to either the lead dev or in the C4 channel.
Q: When should something be a param instead of an InPort?
Whenever it's unlikely to need changing during the running of the pipeline. Examples:
Set nodeNote that for specifying types, it's preferred to just make your script have generic type parameters.
The script resolver will pick up on that and allow the frontend to instantiate it with arbitrary generic
type arguments (from the TypeResolver).
Q: How should I test my integration?
The currently preferred way is to trigger the pipeline off the POST /ws/*/<UUID>/trigger/<Pipeline Name> endpoint.
This endpoint will trace the pipeline, giving you back a JSON array of the state of all nodes' ports
after each pump of the pipeline. Note, though, that many nodes reset their inputs after they run, so Outputs may be
the only non-nulls.