GitLab

We need to track items to remove separately from items to add, otherwise invalid accounts remain in the queue.

The main difference here is that we now remove items from the onlineAccountsImportQueue in a batch, _after_ they have been imported. This prevents duplicates from being added to the queue in the previous time gap between them being removed and imported.

Revert "Allow duplicate variations of each OnlineAccountData in the import queue, but don't allow two entries that match exactly."

This reverts commit 6d9e6e8d.

This caused large gaps with no presence data. They are removed when they expire, causing the local count to drop to zero, and the node would only start requesting them again once a peer had pushed one or more entries proactively.

This also shows that commit 99858f37 now prevents a block candidate with a very small number of online accounts being built immediately after startup.

Fixed Synchronizer.getBlockSummaries() which was expecting BLOCK_SUMMARIES, but updated peers send BLOCK_SUMMARIES_V2

Touches quite a few files because:

* Deprecate HEIGHT_V2 because it doesn't contain enough info to be fully useful during sync.
Newer peers will re-use BLOCK_SUMMARIES_V2.

* For newer peers, instead of sending / broadcasting HEIGHT_V2,
send top N block summaries instead, to avoid requests for minor reorgs.

* When responding to GET_BLOCK, and we don't actually have the requested block,
we currently send an empty BLOCK_SUMMARIES message instead of not responding,
which would cause a slow timeout in Synchronizer.

This pattern has spread to other network message response code,
so now we introduce a generic 'unknown' message type for all these cases.

* Remove PeerChainTipData class entirely and re-use BlockSummaryData instead.

* Each Peer instance used to hold PeerChainTipData - essentially single latest block summary - but now holds a List of latest block summaries.

* PeerChainTipData getter/setter methods modified for compatibility at this point in time.

* Repository methods that return BlockSummaryData (or lists of) now try to fully populate them,
including newly added block reference field.

* Re-worked Peer.canUseCommonBlockData() to be more readable

* Cherry-picked patch to Message.fromByteBuffer() to pass an empty, read-only ByteBuffer to subclass fromByteBuffer() methods, instead of null.
This allows natural use of BufferUnderflowException if a subclass tries to use read(), or hasRemaining(), etc. from an empty data-payload message.
Previously this could have caused an NPE.

Moved various online accounts logs to TRACE level, to make it easier to monitor the queue processing when in DEBUG.

It will now request online accounts every 1 minute instead of every 5 seconds, except for the first 5 minutes following a new online accounts timestamp, in which it will request every 5 seconds (referred to as the "burst" interval). It will also use the burst interval for the first 5 minutes after the node starts.

This is based on the idea that most online accounts arrive soon after a new timestamp begins, and so there is no need to request accounts so frequently after that. This should reduce data usage by a significant amount.

Once mempow is fully rolled out, the "burst" feature can be reduced or removed, since online accounts will be sent ahead of time, generally 15-30 mins prior to the new online accounts timestamp becoming active.

Add accounts from the import queue individually, and then skip future duplicates before unnecessarily validating them again.

This closes a gap where accounts would be moved from onlineAccountsImportQueue to onlineAccountsToAdd, but not yet imported. During this time, there was nothing to stop them from being added to the import queue again, causing duplicate validations.

When validating online accounts, enforce mempow if the online account's timestamp is after the feature trigger.

Allow duplicate variations of each OnlineAccountData in the import queue, but don't allow two entries that match exactly.

This allows some time for initial online account lists to be retrieved, and reduces the chances of the same nonce being computed twice.

Removed online accounts V2 and V1 messaging, as the V3 format will soon be required due to the nonce values.

This will ease the transition to a Q-Chat protocol, where chat messages will no longer be regular transactions.

Added support for ARRR refunds via /crosschain/htlc/refund/{ataddress} and /crosschain/htlc/refundAll

This could probably be refactored into multiple classes to make the code cleaner, but it is functional for now.

Using the feature trigger timestamp here should be much less error prone than a whole new block message version. Once mempow has been live for at least 24 hours, the feature trigger can be removed and the code cleaned up, as all online accounts signatures will use the new format from that time onwards (legacy signatures are trimmed after 24 hours).

Revert "Use block's online accounts timestamp (instead of main timestamp) for the mempow hard fork."

This reverts commit 8cca6db3.

Revert "Use onlineAccountTimestamp for all mempow hard fork related code in OnlineAccountsManager too."

This reverts commit 23423102.