Pause remote index download attempts when rate limited

[mbien]

Detect http 429 from maven central and pause all download attempts.

Testing showed that the index can be downloaded exactly once right now, after that it will return 429.

@neilcsmith-net saw the same yesterday while trying to compare two builds and help testing 3.10 integration.

Details

SEVERE [org.apache.maven.index.updater.DefaultIndexUpdater]: Fallback to *.zip also failed: java.io.FileNotFoundException: Resource nexus-maven-repository-index.zip does not exist
INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager]: Indexing [incremental download, update] of central took 315.72s.
INFO [org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager]: could not (re-)index central
org.apache.maven.wagon.TransferFailedException: transfer failed for https://repo.maven.apache.org/maven2/.index/nexus-maven-repository-index.gz, status: 429
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.backoff(AbstractHttpClientWagon.java:348)
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1197)
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1197)
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1197)
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1197)
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1197)
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1197)
	at org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:1140)
	at org.apache.maven.wagon.StreamWagon.getInputStream(StreamWagon.java:126)
	at org.apache.maven.wagon.StreamWagon.getIfNewer(StreamWagon.java:88)
	at org.apache.maven.wagon.StreamWagon.get(StreamWagon.java:61)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager$WagonFetcher.retrieve(NexusRepositoryIndexManager.java:1152)
Caused: java.io.IOException: Transfer for nexus-maven-repository-index.gz failed; transfer failed for https://repo.maven.apache.org/maven2/.index/nexus-maven-repository-index.gz, status: 429
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager$WagonFetcher.retrieve(NexusRepositoryIndexManager.java:1169)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager$WagonFetcher.retrieve(NexusRepositoryIndexManager.java:1136)
	at org.apache.maven.index.updater.DefaultIndexUpdater.loadIndexDirectory(DefaultIndexUpdater.java:177)
	at org.apache.maven.index.updater.DefaultIndexUpdater$LuceneIndexAdaptor.setIndexFile(DefaultIndexUpdater.java:436)
	at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:604)
	at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:145)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager.indexLoadedRepo(NexusRepositoryIndexManager.java:568)
Caused: java.io.IOException: Failed to load maven-index for: https://repo.maven.apache.org/maven2/
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager.indexLoadedRepo(NexusRepositoryIndexManager.java:602)
[catch] at org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager.lambda$spawnIndexLoadedRepo$12(NexusRepositoryIndexManager.java:712)
	at org.netbeans.modules.openide.util.DefaultMutexImplementation.writeAccess(DefaultMutexImplementation.java:229)
	at org.openide.util.Mutex.writeAccess(Mutex.java:253)
	at org.netbeans.modules.maven.indexer.NexusRepositoryIndexManager.lambda$spawnIndexLoadedRepo$13(NexusRepositoryIndexManager.java:705)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1370)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:45)
	at org.openide.util.lookup.Lookups.executeWith(Lookups.java:287)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1981)

[neilcsmith-net]

LGTM.

I wonder whether we need to special case 429? Probably safer to, although I wonder what exception causes don't justify pausing the download attempts?

[mbien]

I wonder what exception causes don't justify pausing the download attempts?

network issues or something like that?

[neilcsmith-net]

Sure, and they'll probably end up with 429 eventually for retrying! 😄