The strange thing is, in my Lemmy back-end, everything seems normal, and your server is telling mine it’s receiving correctly, but we can see it is not.
/inbox is the endpoint for federation with ActivityPub, so definitely on the right track. Try increasing verbosity to get a full stack traceback to where in the code it’s bugging.
InboxTimeout means your server is taking too long to process API calls to /inbox, I’ve had them before too due to misconfiguration of the reverse proxy. Make sure you’re passing on the right headers, especially the IP headers so they aren’t rate limited and the Accept header so your server knows to send json when requested.
Troubleshooting federation gives many options for testing and checking your configuration, running through them might give your more insight.
Hmmm, some InboxTimeouts are to be expected. The second one could be the result of a scraper or bot. It’s still strange your federation list is empty. I’d take a look at the database, clearing the federation queue (public.federation_queue_state) is a good idea. ‘public.instance’ should be populated with servers (mine has 2836). Peaking into the log files of the database could give some more insight as well.
A successfully federated incoming action will look like this:
2025-07-14T22:40:54.321151Z INFO HTTP request{http.method=POST http.scheme="http" http.host=0d.gs http.target=/inbox otel.kind="server" request_id=61bf0ea2-8df3-42ff-a433-3bec8b37047c}: actix_web::middleware::logger: 10.10.0.1'POST /inbox HTTP/1.1' 2000'-''Lemmy/0.19.12; +https://lemmy.ml/' 0.158242
Your outgoing federation seems to be fine now on my end, this is how my server sees yours:
Lemmy’s docs show how to increase the verbosity of your logs, taking a look at them could point to possibly issues.
Compare https://blog.kaki87.net/api/v3/federated_instances with https://jlai.lu/api/v3/federated_instances and I don’t think your server is federating properly. This graph of instance health shows federation stopped around 20/06.
Since then, your new posts have federated to for example my server and lemmy.world, but 2 new posts from the last week did not make it to my server.
I tried voting on this post with lemmy.world and the same post with 0d.gs which do not seem to federate but are visible on their respective servers.
The strange thing is, in my Lemmy back-end, everything seems normal, and your server is telling mine it’s receiving correctly, but we can see it is not.
What am I looking for in the logs though ? All I see is “InboxTimeout”…
/inbox is the endpoint for federation with ActivityPub, so definitely on the right track. Try increasing verbosity to get a full stack traceback to where in the code it’s bugging.
InboxTimeout means your server is taking too long to process API calls to /inbox, I’ve had them before too due to misconfiguration of the reverse proxy. Make sure you’re passing on the right headers, especially the IP headers so they aren’t rate limited and the Accept header so your server knows to send json when requested.
Troubleshooting federation gives many options for testing and checking your configuration, running through them might give your more insight.
So, here’s the full error :
lemmy-1 | 2025-07-14T00:34:32.098630Z DEBUG HTTP request{http.method=POST http.scheme="https" http.host=blog.kaki87.net http.target=/inbox otel.kind="server" request_id=a9c359f6-eb9d-4888-9ac2-55c1f5bb8e43}: actix_web::middleware::logger: Error in response: LemmyError { message: InboxTimeout, inner: InboxTimeout, context: SpanTrace [{ target: "lemmy_server::root_span_builder", name: "HTTP request", fields: "http.method=POST http.scheme=\"https\" http.host=blog.kaki87.net http.target=/inbox otel.kind=\"server\" request_id=a9c359f6-eb9d-4888-9ac2-55c1f5bb8e43", file: "src/root_span_builder.rs", line: 16 }] } lemmy-1 | 2025-07-14T00:34:32.098834Z WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: InboxTimeout: InboxTimeoutI’ve also been seeing this one :
lemmy-1 | 2025-07-14T00:41:00.407762Z DEBUG HTTP request{http.method=GET http.scheme="http" http.host=lemmy:8536 http.target=/api/v3/site otel.kind="server" request_id=9aab8319-3f3b-4f4e-8530-3b3633afe2f1}: actix_web::extract: Error for Option<T> extractor: IncorrectLogin: IncorrectLogin lemmy-1 | 2025-07-14T00:41:00.427630Z DEBUG HTTP request{http.method=GET http.scheme="http" http.host=lemmy:8536 http.target=/api/v3/post/list otel.kind="server" request_id=ef746ac7-b9a3-4615-bb37-4c12f0ed0827}: actix_web::extract: Error for Option<T> extractor: IncorrectLogin: IncorrectLoginAs for the reverse proxy, I’m using Caddy with the configuration provided here, and it’s supposed to pass those headers by default.
The suggested
curlrequests work.I wish there was a tool that could just tell what’s wrong, like https://federationtester.matrix.org/ for Matrix…
Hmmm, some InboxTimeouts are to be expected. The second one could be the result of a scraper or bot. It’s still strange your federation list is empty. I’d take a look at the database, clearing the federation queue (public.federation_queue_state) is a good idea. ‘public.instance’ should be populated with servers (mine has 2836). Peaking into the log files of the database could give some more insight as well.
A successfully federated incoming action will look like this:
2025-07-14T22:40:54.321151Z INFO HTTP request{http.method=POST http.scheme="http" http.host=0d.gs http.target=/inbox otel.kind="server" request_id=61bf0ea2-8df3-42ff-a433-3bec8b37047c}: actix_web::middleware::logger: 10.10.0.1 'POST /inbox HTTP/1.1' 200 0 '-' 'Lemmy/0.19.12; +https://lemmy.ml/' 0.158242Your outgoing federation seems to be fine now on my end, this is how my server sees yours:
id 2794; ;domain "blog.kaki87.net"; published "2025-06-30T11:35:32.398830Z" ;updated "2025-07-14T00:42:32.758669Z" ;software "lemmy" ;version "0.19.11" ;federation_state: ;instance_id 2794 ;last_successful_id 46037 ;last_successful_published_time "2025-07-14T21:12:27.481507Z" ;fail_count 0You could try updating or reinstalling your back-end as well, lemmy is at 0.19.12.