Akkoma stable 2023.03 - recently drafted purge requests

a small little release for you today, nothing huge, a few bug fixes and some cool little frontend thingies

New Stuff

Post drafts

This is very hard to capture in a screenshot ~_~

image

basically, you know when you type a reply to a post, and you accidentally navigate away/close the post form?

well now your hard-typed post will not disappear, but will instead remain for you to find again where you left it! hooray

Recently Used Emojo

image

now this one is easy to illustrate in a screenshot. use a lot of emoji? use the same few a lot and have to continuously type the shortcode?

no longer!

now when you use an emoji, it is stored in your list of recently used emoji for later use

The great activity purger

before:


after:

We’ve got an object pruner, right? Nice and simple, deletes objects older than x days.

Problem is that every object has one or many activities that reference it, and they can really build up.

Older instances probably have 10-20GB of activities laying around, that are borderline-useless.

Now there’s a solution - the activity pruner

Check out https://docs.akkoma.dev/develop/administration/CLI_tasks/database/#options_1 - we’ve now got --prune-orphaned-activities which will also prune the activities table.

WARNING: this will lead to irreversable data loss, especially if you are not careful with the flags you provide. That is the intention, but make sure you double-check before you hit run.

WARNING THE SECOND: this is massively database-intensive. if you are an older instance and plan to do this, i’d advise making a thread in support, you’ll likely want to slowly delete activities in the background

Links

As ever, updating your instance

2023.02 changelog

Thankies

Go out to:

  • Mahito TANNO, quad for doc errata fixing
  • YhĂ«htozr for making us able to display the BIGGE misskey emoji
  • flisk for errata and frontend stuff - the recent emoji thing is really nice!
  • foxing for helping negotiate functionality with clients
  • ilja for going once again going absolutely too far into database pruning, it works really nicely - praise having disk space
  • solidsanek for post draft functionality - finally i won’t lose my essay of incoherent rambling :hehe:
6 Likes

the OTP build was bugged on 3.7.0, 3.7.1 is currently building which will fix that issue

it’s known, i’ll edit this when it’s finished

Edit: it is done, it’s safe to upgrade

4 Likes

Thanks :heart:

Does the activity pruner only prune remote activities?

Short answer: No, but that’s not really relevant for activities, only for (non-activity) objects.

Long answer:
It’s a bit technical, but I think it’s important to understand the difference between Object and Activity here (well, technically speaking Activities are also Objects, but let’s ignore that for now).

When we talk about objects in the context of this pruner, we talk about things like Note, Article, Question… These are what you see as “posts” on your timeline.

Activities are things that act on these objects. Create, Like (ie favourite) , Announce (ie repeat), Answer (ie vote on a poll), Flag (ie report a post)…

The prune task prunes objects. But this new parameter allows you to also remove what I call “orphaned activities”, which are just activities without corresponding objects.

For example, if someone on a remote instance made a post, that will result in a Create activity and a Note object. If you favourite that post, it will result in a Like activity referencing that Note. When the Note is deleted (maybe you pruned it, maybe the person who made the note deleted it, etc.), the Note object will be gone.

So now the reasoning is that, at this point, the activities on this object aren’t relevant any more, since they act on an object that doesn’t exist. So the --prune-orphaned-activities option will delete them. It doesn’t differentiate between local or remote, because that’s not really relevant here.

To go a bit further:
Local vs remote is relevant, however, for pruning objects. If you want to keep your local activities, then it makes sense that you want to keep the corresponding objects as well. In that case, you can use the --keep-threads option. This will keep the objects that are part of a thread that have local activities on them.

To give an example:
Let’s say someone remote made a post and you favourite it.

When you prune the objects with ./bin/pleroma_ctl database prune_objects --prune-orphaned-activities, it will delete that post, and the --prune-orphaned-activities option will then cause your favourite to also be deleted.

When you prune the objects with ./bin/pleroma_ctl database prune_objects --keep-threads --prune-orphaned-activities, it will see that the post has local interaction, and thus keep the post. The --prune-orphaned-activities option will keep your favourite because the corresponding object wasn’t deleted. (Unless the post was deleted in another way ofc, then the object won’t exist any more either, and thus the favourite will still be removed from the database.)

Personally I run prune_objects with both the --keep-threads and --prune-orphaned-activities options on a weekly basis.

3 Likes

Oof, that’s a big post :sweat_smile:

I know it’s quite complex that way, so if someone has ideas on making this better/easier to grasp/easier to understand/… pls do share it. I think this can really help in keeping database sizes smaller without removing too much, so the more accessible it is to people, the better I think.

3 Likes

My understanding of pruning (please correct if wrong)

  1. By default, local objects - not activities - are kept forever, not touched by pruner
  2. Remote objects are safe to prune because they can be re-fetched on demand
  3. --prune-orphaned-activities may prune local activities, because it performs one check only, “does associated object exist in postgres?”

I’d expand a bit, but I think you’re generally correct, yes

  1. By default, local objects are kept forever, not touched by pruner.
  2. By default, all activities are kept forever, not touched by pruner.
  3. Remote objects are generally safe to prune because they can be re-fetched on demand (but can still lead to eg broken threads. I also think that only public posts can be refetched, not direct or followers-only).
  4. Last point is indeed correct: --prune-orphaned-activities may prune local activities, because it performs one check only, “does associated object exist in postgres?”
1 Like

point 3 is important if you or people in your bubble use followers-only posts a lot, since they cannot be refetched (this is the mistake I made)

2 Likes

is it a fixable one?

no, if you delete non-public posts they are gone forever
however that’s not too big a deal depending on how much you care about historic posts

1 Like

Woooo! Just finished running the --prune-orphaned-activities database job and my DB went from 48GB to 7GB.

this is your brain on following too many relays

3 Likes