hi migrating akkoma on new debian bookworm system, running into an error with install frontend:
akkoma@z133:~$ ./bin/pleroma_ctl frontend install pleroma-fe --ref stable
--rpc-eval : RPC failed with reason :noconnection
I thought it might be network related however the vps is online and pinging.
any support is appreciated.
is Akkoma running? (if not start it)
akkoma is running. I sttarted it from daemon and with service akkoma start same result.
ok now it works, however it throws up a nnew error:
** (ArgumentError) errors were found at the given arguments:
* 1st argument: the table identifier does not refer to an existing ETS table
(stdlib 5.0.2) :ets.lookup(:elixir_config, :on_undefined_variable)
(elixir 1.15.4) src/elixir_config.erl:21: :elixir_config.get/1
(elixir 1.15.4) src/elixir_env.erl:49: :elixir_env.set_prematch_from_config/1
(elixir 1.15.4) lib/code.ex:543: Code.validated_eval_string/3
not sure what is going on!
ok it looks like the akkoma service is running then stopping quite quickly, since within a moment the fontend install command goes to >rpc failed with reason noconnection afterr providing the error posted in the previous message.
the update and migration went fairly smoothly I had to chown some file to get it to run correctly however after that there were no errors.
edit>
yeah I checked the log and akkoma is erroring out!
May 09 08:54:03 z133 systemd[1]: akkoma.service: Consumed 3.216s CPU time.
May 09 08:54:03 z133 systemd[1]: akkoma.service: Failed with result 'exit-code'.
May 09 08:54:03 z133 systemd[1]: akkoma.service: Main process exited, code=exited, status=1/FAILURE
May 09 08:54:02 z133 pleroma[15218]: Crash dump is being written to: erl_crash.dump...done
May 09 08:53:58 z133 pleroma[15218]:
May 09 08:53:58 z133 pleroma[15218]: Kernel pid terminated (application_controller) ({application_start_failure,pleroma,{bad_return,{{'Elixir.Pleroma.Appli>
May 09 08:53:56 z133 pleroma[15218]: 08:53:56.807 [notice] :alarm_handler: {:clear, :system_memory_high_watermark}
May 09 08:53:56 z133 pleroma[15218]: (kernel 9.0.2) application_master.erl:293: :application_master.start_it_old/4
May 09 08:53:56 z133 pleroma[15218]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/application.ex:53: Pleroma.Application.start/2
May 09 08:53:56 z133 pleroma[15218]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/config/deprecation_warnings.ex:223: Pleroma.Config.DeprecationWarn>
May 09 08:53:56 z133 pleroma[15218]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/config/deprecation_warnings.ex:398: Pleroma.Config.DeprecationWarn>
May 09 08:53:56 z133 pleroma[15218]: ** (ArgumentError) No base_url set for uploads - please set one in your config!
May 09 08:53:56 z133 pleroma[15218]: ** (EXIT) an exception was raised:
May 09 08:53:56 z133 pleroma[15218]: 08:53:56.803 [notice] Application pleroma exited: exited in: Pleroma.Application.start(:normal, [])
May 09 08:53:56 z133 pleroma[15218]: Please note that it is HEAVILY recommended to use a subdomain to host user-uploaded media!
May 09 08:53:56 z133 pleroma[15218]: * `config :pleroma, Pleroma.Upload, base_url: "https://example.com/media/`
May 09 08:53:56 z133 pleroma[15218]: Please make the following change:
May 09 08:53:56 z133 pleroma[15218]: Your config does not specify a base_url for uploads!
May 09 08:53:56 z133 pleroma[15218]: 08:53:56.802 [error] !!!WARNING!!!
May 09 08:53:56 z133 pleroma[15218]: 08:53:56.549 [notice] :alarm_handler: {:set, {:system_memory_high_watermark, []}}
May 09 08:53:52 z133 systemd[1]: Started akkoma.service - Akkoma social network.
May 09 08:53:52 z133 systemd[1]: akkoma.service: Consumed 2.924s CPU time.
May 09 08:53:52 z133 systemd[1]: Stopped akkoma.service - Akkoma social network.
May 09 08:53:52 z133 systemd[1]: akkoma.service: Scheduled restart job, restart counter is at 89.
May 09 08:53:52 z133 systemd[1]: akkoma.service: Consumed 2.924s CPU time.
May 09 08:53:52 z133 systemd[1]: akkoma.service: Failed with result 'exit-code'.
May 09 08:53:52 z133 systemd[1]: akkoma.service: Main process exited, code=exited, status=1/FAILURE
May 09 08:53:52 z133 pleroma[15094]: Crash dump is being written to: erl_crash.dump...done
May 09 08:53:52 z133 pleroma[15094]:
May 09 08:53:52 z133 pleroma[15094]: Kernel pid terminated (application_controller) ({application_start_failure,pleroma,{bad_return,{{'Elixir.Pleroma.Appli>
May 09 08:53:50 z133 pleroma[15094]: 08:53:50.857 [notice] :alarm_handler: {:clear, :system_memory_high_watermark}
May 09 08:53:50 z133 pleroma[15094]: (kernel 9.0.2) application_master.erl:293: :application_master.start_it_old/4
May 09 08:53:50 z133 pleroma[15094]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/application.ex:53: Pleroma.Application.start/2
May 09 08:53:50 z133 pleroma[15094]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/config/deprecation_warnings.ex:223: Pleroma.Config.DeprecationWarn>
May 09 08:53:50 z133 pleroma[15094]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/config/deprecation_warnings.ex:398: Pleroma.Config.DeprecationWarn>
May 09 08:53:50 z133 pleroma[15094]: ** (ArgumentError) No base_url set for uploads - please set one in your config!
May 09 08:53:50 z133 pleroma[15094]: ** (EXIT) an exception was raised:
May 09 08:53:50 z133 pleroma[15094]: 08:53:50.850 [notice] Application pleroma exited: exited in: Pleroma.Application.start(:normal, [])
May 09 08:53:50 z133 pleroma[15094]: Please note that it is HEAVILY recommended to use a subdomain to host user-uploaded media!
May 09 08:53:50 z133 pleroma[15094]: * `config :pleroma, Pleroma.Upload, base_url: "https://example.com/media/`
May 09 08:53:50 z133 pleroma[15094]: Please make the following change:
May 09 08:53:50 z133 pleroma[15094]: Your config does not specify a base_url for uploads!
May 09 08:53:50 z133 pleroma[15094]: 08:53:50.849 [error] !!!WARNING!!!
May 09 08:53:50 z133 pleroma[15094]: 08:53:50.641 [notice] :alarm_handler: {:set, {:system_memory_high_watermark, []}}
any support is appreciated.
As mentioned before you should read all reelase announcments since 3.9.3. The error already tells you that you forgot to update your config accordingly and thus it refuses to start
oh I read up to the update telling to update to bookworm debian I will look again. sorry to bother!
where do I find the config mentioned here to change the media upload url?
it is not specified in the changelog.
https://meta.akkoma.dev/t/akkoma-stable-2024-03-securer-i-barely-know-her/681#explicit-upload-and-media-proxy-domains-5
(don’t try to update the config and migrate to a subdomain at the same time; first just keep the current domain and make sure the update works; then migrate the domain later. A migration guide is also linked in the announcmenet linked above)
thanks for your support with this. I do not understand how to define a base_url, is there a config file somewhere to edit, or is this a command line instruction?
thanks again.
please read your logs - it tells you explicitly how to do it
sorry I do not understand, thhere is no “`config” command.
entering this from terminal does not appear to work:
`config :pleroma, Pleroma.Upload, base_url: "https://fedi.lamer-ethos.site/media/"
sorry again to trouble you I am not seeing how to do this.
you must have installed this at some point - you will have created a config.exs - it’s that.
oh I see it! sorry I had not had experience editing the akkoma config.
thanks for your help!
now it works! confirmed my akkoma install is now updated to 3.13.1!
thanks to FloatingGhost and Oneric for your help.
sorry I did not know about editing the config I had not had to do that before!
sorry to bug you all again however now my friends timeline is not loading.
recent log output:
May 09 13:17:08 z133 pleroma[185189]: 13:17:08.277 request_id=F82tKa82UUlQCrMAAHGB [error] Could not fetch user https://gametoots.de/users/totorruns, {nil,>
May 09 13:17:08 z133 pleroma[185189]: 13:17:08.277 request_id=F82tKa82UUlQCrMAAHGB [error] Could not decode user at fetch https://gametoots.de/users/totorr>
May 09 13:17:07 z133 pleroma[185189]: 13:17:07.966 request_id=F82tKa82UUlQCrMAAHGB [error] Could not fetch user https://gametoots.de/users/totorruns, {nil,>
May 09 13:17:07 z133 pleroma[185189]: 13:17:07.966 request_id=F82tKa82UUlQCrMAAHGB [error] Could not decode user at fetch https://gametoots.de/users/totorr>
May 09 13:17:06 z133 pleroma[185189]: 13:17:06.621 [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
May 09 13:17:03 z133 pleroma[185189]: 13:17:03.034 [error] Object rejected while fetching https://mastodon.nz/users/SimonCHulse/statuses/112408488828560009>
May 09 13:17:01 z133 pleroma[185189]: 13:17:01.738 [error] Object rejected while fetching https://mastodon.nz/users/draftexcluder/statuses/1120637996560838>
May 09 13:17:00 z133 pleroma[185189]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex:5: Pleroma.Web.Ma>
May 09 13:17:00 z133 pleroma[185189]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex:71: Pleroma.Web.M>
May 09 13:17:00 z133 pleroma[185189]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/web/activity_pub/activity_pub.ex:537: Pleroma.Web.ActivityPub.Activit>
May 09 13:17:00 z133 pleroma[185189]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/pagination.ex:40: Pleroma.Pagination.fetch_paginated/4
May 09 13:17:00 z133 pleroma[185189]: (ecto 3.10.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
May 09 13:17:00 z133 pleroma[185189]: (ecto 3.10.3) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
May 09 13:17:00 z133 pleroma[185189]: (ecto_sql 3.10.2) lib/ecto/adapters/sql.ex:945: Ecto.Adapters.SQL.execute/6
May 09 13:17:00 z133 pleroma[185189]: (ecto_sql 3.10.2) lib/ecto/adapters/sql.ex:1047: Ecto.Adapters.SQL.raise_sql_call_error/1
May 09 13:17:00 z133 pleroma[185189]: ** (DBConnection.ConnectionError) tcp recv: closed (the connection was closed by the pool, possibly due to a time>
May 09 13:17:00 z133 pleroma[185189]: ** (exit) an exception was raised:
May 09 13:17:00 z133 pleroma[185189]: Request: GET /api/v1/timelines/home?with_muted=true&reply_visibility=following&limit=20
May 09 13:17:00 z133 pleroma[185189]: Server: fedi.lamer-ethos.site:80 (http)
May 09 13:17:00 z133 pleroma[185189]: 13:17:00.300 [error] #PID<0.6363.0> running Pleroma.Web.Endpoint (connection #PID<0.6360.0>, stream id 1) terminated
May 09 13:17:00 z133 pleroma[185189]: 13:17:00.300 request_id=F82tJGZdmAmuZIQAAOuC [error] Internal server error: %DBConnection.ConnectionError{message: "t>
May 09 13:17:00 z133 pleroma[185189]: (db_connection 2.6.0) lib/db_connection.ex:743: DBConnection.parsed_prepare_execute/5
May 09 13:17:00 z133 pleroma[185189]: (db_connection 2.6.0) lib/db_connection.ex:1607: DBConnection.run/6
May 09 13:17:00 z133 pleroma[185189]: (db_connection 2.6.0) lib/db_connection.ex:1512: DBConnection.run_execute/5
May 09 13:17:00 z133 pleroma[185189]: (db_connection 2.6.0) lib/db_connection/holder.ex:354: DBConnection.Holder.holder_apply/4
May 09 13:17:00 z133 pleroma[185189]: (postgrex 0.17.5) lib/postgrex/protocol.ex:2077: Postgrex.Protocol.bind_execute_close/4
May 09 13:17:00 z133 pleroma[185189]: (postgrex 0.17.5) lib/postgrex/protocol.ex:2222: Postgrex.Protocol.recv_bind/3
May 09 13:17:00 z133 pleroma[185189]: (postgrex 0.17.5) lib/postgrex/protocol.ex:3197: Postgrex.Protocol.msg_recv/4
May 09 13:17:00 z133 pleroma[185189]: :prim_inet.recv0/3
May 09 13:17:00 z133 pleroma[185189]: #PID<0.6363.0> was at location:
May 09 13:17:00 z133 pleroma[185189]: 13:17:00.294 [error] Postgrex.Protocol (#PID<0.3948.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0>
May 09 13:16:59 z133 pleroma[185189]: 13:16:59.402 request_id=F82tJ3HGOSHEsbUAAG2B [error] Could not fetch user https://trollian.space/users/airfilter, {ni>
May 09 13:16:59 z133 pleroma[185189]: 13:16:59.402 request_id=F82tJ3HGOSHEsbUAAG2B [error] Could not decode user at fetch https://trollian.space/users/airf>
May 09 13:16:59 z133 pleroma[185189]: 13:16:59.220 request_id=F82tJ3HGOSHEsbUAAG2B [error] Could not fetch user https://trollian.space/users/airfilter, {ni>
May 09 13:16:59 z133 pleroma[185189]: 13:16:59.220 request_id=F82tJ3HGOSHEsbUAAG2B [error] Could not decode user at fetch https://trollian.space/users/airf>
May 09 13:16:52 z133 pleroma[185189]: 13:16:52.502 [error] Object rejected while fetching https://flipboard.social/users/CultureDesk/statuses/1124078510096>
May 09 13:16:52 z133 pleroma[185189]: 13:16:52.502 [error] Object rejected while fetching https://flipboard.social/users/CultureDesk/statuses/1124078510096>
May 09 13:16:48 z133 pleroma[185189]: 13:16:48.118 [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
May 09 13:16:46 z133 pleroma[185189]: 13:16:46.881 request_id=F82tJH2b6vM9J0AAAGwB [error] Could not decode user at fetch https://infosec.exchange/users/ca>
May 09 13:16:45 z133 pleroma[185189]: 13:16:45.698 request_id=F82tJH2b6vM9J0AAAGwB [error] Could not decode user at fetch https://infosec.exchange/users/ca>
May 09 13:16:34 z133 pleroma[185189]: 13:16:34.443 [error] Object rejected while fetching https://flipboard.social/users/CultureDesk/statuses/1124078510096>
May 09 13:16:34 z133 pleroma[185189]: 13:16:34.443 [error] Object rejected while fetching https://flipboard.social/users/CultureDesk/statuses/1124078510096>
May 09 13:16:30 z133 pleroma[185189]: 13:16:30.121 [notice] TLS :client: In state :connection received SERVER ALERT: Fatal - Close Notify
May 09 13:16:29 z133 pleroma[185189]: 13:16:29.869 [notice] TLS :client: In state :connection received SERVER ALERT: Fatal - Close Notify
May 09 13:16:23 z133 pleroma[185189]: 13:16:23.394 request_id=F82tG-D-bnrNso8AAFwB [error] Internal server error: %Postgrex.Error{message: nil, postgres: %>
May 09 13:16:23 z133 pleroma[185189]: (db_connection 2.6.0) lib/db_connection.ex:743: DBConnection.parsed_prepare_execute/5
May 09 13:16:23 z133 pleroma[185189]: (db_connection 2.6.0) lib/db_connection.ex:1607: DBConnection.run/6
May 09 13:16:23 z133 pleroma[185189]: (db_connection 2.6.0) lib/db_connection.ex:1512: DBConnection.run_execute/5
the page shows “error fetching timeline: internal server error”.
is it a dbconnection connectionerror? the ‘all’ timeline loads fine.
any support is appreciated. I did not like to start a new topic for this issue since it could be a continuation of the upgrade issues I had.
thanks!
ay 09 13:17:00 z133 pleroma[185189]: 13:17:00.300 request_id=F82tJGZdmAmuZIQAAOuC [error] Internal server error: %DBConnection.ConnectionError{message: "t>
hard to know since you’ve cut off the actual message
journalctl unfortuntanetly likes to truncate long lines by default; try journalctl -u akkoma | less +G in the future to prevent this; you can scroll up/down with arrow keys and exit this view with q.
The actual errors are truncated, but this part
** (DBConnection.ConnectionError) tcp recv: closed (the connection was closed by the pool, possibly due to a time
hints at your queries timing out or otherwise loosing connection to your database. I doubt this has something to do with the update, but rather is a return of your initial timeout issue.
Timeline queries are relatively expensive and the friends timeline has to sieve through more posts than “all”, so it timing out first makes sense this time.
I’ll make a guess and presume this actually is the issue in the following.
Note your prior logs from missing base URL also again indicated most of the systems memory is already used by other processes. Even without actually hitting OOM, this will degrade performance as the system needs to do more work to organize and compact pages and less to no RAM is available for file page caching meaning more, slow disk accesses. You’re putting more load on your hardware than it can handle well.
There’s a chance pruning old remote objects and activities and reindexing helps somewhat, but the prune itself also takes a fair bit of resources and more RAM than usual (and at least on large instances can take multiple days). Since you appear to already push hardware capabilities to the limit just during regular operation, this seems problematic
the only active service I have currently running on this vps is akkoma, which takes up around 1 gig+ of memory. (I also run owncast however that is very low resource unless it is serving a stream)
I pruned all posts older than one day however that did not help the current issue.
with the journalctl / less command Oneric posted, the tcp error is now complete:
May 10 11:29:57 z133 pleroma[614]: 11:29:57.540 request_id=F8315Q6kM3uU-m4AAUSC [error] Could not fetch user https://gametoots.de/users/totorruns, {nil, {:error, :not_found}}
May 10 11:29:57 z133 pleroma[614]: 11:29:57.840 request_id=F8315Q6kM3uU-m4AAUSC [error] Could not decode user at fetch https://gametoots.de/users/totorruns, :not_found
May 10 11:29:57 z133 pleroma[614]: 11:29:57.840 request_id=F8315Q6kM3uU-m4AAUSC [error] Could not fetch user https://gametoots.de/users/totorruns, {nil, {:error, :not_found}}
May 10 11:30:00 z133 pleroma[614]: 11:30:00.319 [error] Object rejected while fetching https://universeodon.com/users/blackops/statuses/112224588594553867#delete {:fetch, {:error, :not_found}}
May 10 11:30:17 z133 pleroma[614]: 11:30:17.835 request_id=F8316aVSDyrh2IQAAUkC [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
May 10 11:30:18 z133 pleroma[614]: 11:30:18.704 request_id=F8316aVSDyrh2IQAAUkC [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
May 10 11:30:19 z133 pleroma[614]: 11:30:19.210 [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
May 10 11:30:20 z133 pleroma[614]: 11:30:20.352 [error] Postgrex.Protocol (#PID<0.3945.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.7422.0> timed out because it queued and checked out the connection for longer than 15000ms
May 10 11:30:20 z133 pleroma[614]: #PID<0.7422.0> was at location:
May 10 11:30:20 z133 pleroma[614]: :prim_inet.recv0/3
May 10 11:30:20 z133 pleroma[614]: (postgrex 0.17.5) lib/postgrex/protocol.ex:3197: Postgrex.Protocol.msg_recv/4
May 10 11:30:20 z133 pleroma[614]: (postgrex 0.17.5) lib/postgrex/protocol.ex:2222: Postgrex.Protocol.recv_bind/3
May 10 11:30:20 z133 pleroma[614]: (postgrex 0.17.5) lib/postgrex/protocol.ex:2077: Postgrex.Protocol.bind_execute_close/4
May 10 11:30:20 z133 pleroma[614]: (db_connection 2.6.0) lib/db_connection/holder.ex:354: DBConnection.Holder.holder_apply/4
May 10 11:30:20 z133 pleroma[614]: (db_connection 2.6.0) lib/db_connection.ex:1512: DBConnection.run_execute/5
May 10 11:30:20 z133 pleroma[614]: (db_connection 2.6.0) lib/db_connection.ex:1607: DBConnection.run/6
May 10 11:30:20 z133 pleroma[614]: (db_connection 2.6.0) lib/db_connection.ex:743: DBConnection.parsed_prepare_execute/5
May 10 11:30:20 z133 pleroma[614]: 11:30:20.393 request_id=F8315uqoUPtG8ioAAUYC [error] Internal server error: %DBConnection.ConnectionError{message: "tcp recv: closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been terminated)", severity: :error, reason: :error}
May 10 11:30:20 z133 pleroma[614]: 11:30:20.393 [error] #PID<0.7422.0> running Pleroma.Web.Endpoint (connection #PID<0.7420.0>, stream id 1) terminated
May 10 11:30:20 z133 pleroma[614]: Server: fedi.lamer-ethos.site:80 (http)
May 10 11:30:20 z133 pleroma[614]: Request: GET /api/v1/timelines/home?with_muted=true&reply_visibility=following&limit=20
May 10 11:30:20 z133 pleroma[614]: ** (exit) an exception was raised:
May 10 11:30:20 z133 pleroma[614]: ** (DBConnection.ConnectionError) tcp recv: closed (the connection was closed by the pool, possibly due to a timeout or because the pool has been terminated)
May 10 11:30:20 z133 pleroma[614]: (ecto_sql 3.10.2) lib/ecto/adapters/sql.ex:1047: Ecto.Adapters.SQL.raise_sql_call_error/1
May 10 11:30:20 z133 pleroma[614]: (ecto_sql 3.10.2) lib/ecto/adapters/sql.ex:945: Ecto.Adapters.SQL.execute/6
May 10 11:30:20 z133 pleroma[614]: (ecto 3.10.3) lib/ecto/repo/queryable.ex:229: Ecto.Repo.Queryable.execute/4
May 10 11:30:20 z133 pleroma[614]: (ecto 3.10.3) lib/ecto/repo/queryable.ex:19: Ecto.Repo.Queryable.all/3
May 10 11:30:20 z133 pleroma[614]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/pagination.ex:40: Pleroma.Pagination.fetch_paginated/4
May 10 11:30:20 z133 pleroma[614]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/web/activity_pub/activity_pub.ex:537: Pleroma.Web.ActivityPub.ActivityPub.fetch_activities/3
May 10 11:30:20 z133 pleroma[614]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex:71: Pleroma.Web.MastodonAPI.TimelineController.home/2
May 10 11:30:20 z133 pleroma[614]: (pleroma 3.13.1-0-gc02e343) lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex:5: Pleroma.Web.MastodonAPI.TimelineController.action/2
May 10 11:30:27 z133 pleroma[614]: 11:30:27.844 [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
May 10 11:30:30 z133 pleroma[614]: 11:30:30.162 [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
May 10 11:30:32 z133 pleroma[614]: 11:30:32.425 [error] Object rejected while fetching https://universeodon.com/users/blackops/statuses/112224588594553867#delete {:fetch, {:error, :not_found}}
May 10 11:30:37 z133 pleroma[614]: 11:30:37.921 [error] Could not decode user at fetch https://infosec.exchange/users/caffeine, :not_found
any suggestions on how to resolve would be greatly appreciated.
taking over 15 seconds to gather a limit of 20 records means that your database is massively struggling - check your I/O with iotop to make sure nothing is hogging your throughput.
you can also check /phoenix/live_dashboard/ecto_stats to make sure your cache hitrate isn’t unreasonably lot