Add support for PageFile free page truncation

[cshannon]

This commit adds support for index compaction using a truncation strategy as described in #2232. See that issue for more information.

Compaction works by reclaiming space by truncating free pages at the end of the file. A large amount of free pages can be allocated during certain use cases (such as a large message backlogs) and this strategy helps by removing the excess space when possible.

The amount of space reclaimed is configurable using a min/max free page ratio. The min free page ratio defines the amount of free pages to leave and the max free page ratio defines the max free pages to allow before attemping to reclaim space.

The compaction check will be performed during the normal checkpoint cycle.

It's also worth noting this doesn't introduce any extra risk of data loss. By default the recovery file is on, and will be able to recover free pages if a failure happens during truncation. In the worst case scenario where the file is completely corrupted (which is the same risk as any other index write, there isn't anything special about truncation), the index corruption is detected and the file can be rebuilt form the journal.

[cshannon]

I added tests here for the PageFile to cover different scenarios, but the test coverage could still probably be expanded. We could add more tests using KahaDB (there's only one now). I am targeting this for 6.4.0 for now, so there is some time as this won't get merged until 6.3.0 is done.

[mattrpav]

@cshannon this is a super awesome feature! First glance, I think it should include some metrics for tracking time, pages and/or file size compacted.

I’m happy to help add those

[cshannon]

@cshannon this is a super awesome feature! First glance, I think it should include some metrics for tracking time, pages and/or file size compacted.

I’m happy to help add those

Metrics for pages/size are useful but time would not be for truncation. Time would be useful for when defrag is implemented but for truncation it's nearly instant time so there's nothing to really measure. Also, something to discuss is if we enable this by default. For something like defragmentation we could leave it off by default, but for a simple truncation like this PR is doing I think having it on by default could make sense. We would need to decide on default parameter thresholds though. For now I picked .1 for minFreePageCompactionRatio (leave 10% free pages relative to the current size after compaction) and .3 for maxFreePageCompactionRatio (don't start trying to compact until there's at least 30% pages that are free relative to the current size)

[cshannon]

Also, this can be configured using spring as long as the value matches the exact enum value, including case:

<persistenceAdapter>
    <kahaDB directory="target/data/kahadb" indexCompactionStrategy="NEVER"/>
</persistenceAdapter>

If we wanted to make it case insensitive we could make the setIndexCompactionStrategy() setter take a String and do the conversion from that manually (we do this for journalDiskSyncStrategy)

[mattrpav]

Specifics to requested changes (I'm happy to add on a feature branch, let me know your preference).

Metrics:

1. Add a min/max/avg page file size metrics (measures effectiveness of enabling trunaciton)
2. Add currentPageFileSize
3. Add trunactionCount <-- count how many times truncation has occurred

Config/Status:

1. truncationSupported: true|false <-- does the fs support truncation?
2. truncationEnabled: true| false <-- did admin set truncation to be on?
3. truncationStatus: active | disabled <-- the resolved value of (supported && enabled)

[cshannon]

I think for the metrics we could do a follow on PR for it. One thing is to keep in mind that we might be expanding (ie defrag) with other compaction options so the naming should be flexible enough. So I agree the naming should probably involve "truncation" because defrag will be different. Defrag would have have its own metrics (like how many pages were moved and how often and how long it took). Ideally we'd run a defrag to move pages, and then future GC cycles could just truncate once the pages got moved.