As an extension to the fully-decentralised learning approach on a DAG, we adjusted the random tip selection procedure to no longer be random. Instead, clients evaluate potential next models during the walk thorugh the DAG, always taking the route of the better-performing model. The rest of the approach stays the same. This way, if clients possess very different data, they implicitly build clusters on the DAG that correspond to specialised models for their data cluster. This approach is more beneficial than splitting the clients before training, because firstly, it may not be possible to split the clients due to data privacy issues, and secondly, in the beginning of the training process, the clusters do not build yet, because the everyone can benefit from each others updates. Thus, the models for each cluster can perform better than models that were trained using only the cluster-specific clients in the first place.
Resilience against Poisoning Attacks
As another benefit, the so-called accuracy tip selection procedure also improves the resilience against model poisoning attacks. They describe an attack where an adversary poses as a benign participant of a learning system, but instead of submitting useful models, they submit completely random ones, of even ones that are trained for a specific misclassification. Due to the accuracy tip selection, these malicious models are very rarely selected and included into the DAG, so that they get stale quickly and do not count towards the network consensus.