Prioritize announcement of pin roots

[lidel]

Hopefully inexpensive optimization that would help Collab cluster, and content routing based on Amino DHT in general:

• before announcing everything, make a quick pass and announce only the root blocks for all pins first.

This ensures most important CIDs are out of the door (in the spirit of ipfs/specs#462), and we can slowly announce everything else.

[aschmahmann]

Seems reasonable. The main tradeoff here is that we might over-read data from disk which may/may not be that important.

The reprovider strategies are here

kubo/core/node/provider.go

Lines 130 to 139 in e22f47a

	switch reprovideStrategy {
	case "all", "":
	keyProvider = fx.Provide(provider.NewBlockstoreProvider)
	case "roots":
	keyProvider = fx.Provide(pinnedProviderStrategy(true))
	case "pinned":
	keyProvider = fx.Provide(pinnedProviderStrategy(false))
	default:
	return fx.Error(fmt.Errorf("unknown reprovider strategy %q", reprovideStrategy))
	}

Probably the easiest way to do this is:

• "roots" -> keep doing the same thing
• "pinned" -> do a first pass using the "roots" function and then do this function (maybe keep something like a bloom filter or map so we don't readvertise the same data twice in exchange for some more memory)
  • If the plan is to use a map here it'd probably be better to rewrite the function to enumerate the roots and then put them into a queue so we don't have to hit the datastore twice
• "all" -> do a first pass using the "roots" function, then do the blockstore enumeration
  • Could do roots -> pinned -> all but if most of the data is pinned it'll require enumerating the blockstore twice which could be quite expensive. This way the worst case is if the blockstore is lots of tiny objects in which case the datastore and blockstore are both enumerated (might still be annoying, but likely less bad)

[aschmahmann]

From conversation with @lidel:

Something we could also do here is doing a local walk (i.e. don't do any remote fetching) of MFS and announcing the file/folder roots first (along with the other "roots").

Tracked in #10386