borg 0.24+ (normal, with fadvise DONTNEED)
==========================================

following should be slow:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 12.934 s, 415 MB/s

real    0m12.937s
user    0m0.008s
sys 0m1.986s

following should be fast:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 0.773392 s, 6.9 GB/s

real    0m0.774s
user    0m0.005s
sys 0m0.769s

borg backups 8 1GB files...
Initializing repository at "repo"
Encryption NOT enabled.
Use the "--encryption=repokey|keyfile|passphrase" to enable encryption.
Synchronizing chunks cache...
Rebuilding archive collection. Known: 0 Repo: 0 Unknown: 0
Merging collection into master chunks cache...
Done.

if following is slow, borg killed the cache:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 0.707791 s, 7.6 GB/s  <-- fast

real    0m0.709s
user    0m0.005s
sys 0m0.703s

following should be slow:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 11.6848 s, 459 MB/s

real    0m11.687s
user    0m0.012s
sys 0m1.916s

following should be fast:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 0.812345 s, 6.6 GB/s

real    0m0.813s
user    0m0.005s
sys 0m0.808s

borg backups 8 1GB files...
Initializing repository at "repo"
Encryption NOT enabled.
Use the "--encryption=repokey|keyfile|passphrase" to enable encryption.
Synchronizing chunks cache...
Rebuilding archive collection. Known: 0 Repo: 0 Unknown: 0
Merging collection into master chunks cache...
Done.

if following is slow, backing up the 5GB file removed it from the cache:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 11.5622 s, 464 MB/s   <-- slow

real    0m11.563s
user    0m0.011s
sys 0m1.931s


borg 0.24+ (patch, without fadvise DONTNEED)
============================================

following should be slow:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 12.969 s, 414 MB/s

real    0m12.971s
user    0m0.009s
sys 0m1.945s

following should be fast:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 0.785915 s, 6.8 GB/s

real    0m0.787s
user    0m0.005s
sys 0m0.782s

borg backups 8 1GB files...
Initializing repository at "repo"
Encryption NOT enabled.
Use the "--encryption=repokey|keyfile|passphrase" to enable encryption.
Synchronizing chunks cache...
Rebuilding archive collection. Known: 0 Repo: 0 Unknown: 0
Merging collection into master chunks cache...
Done.

if following is slow, borg killed the cache:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 6.07772 s, 883 MB/s    <-- "slow"

real    0m6.084s
user    0m0.007s
sys 0m1.241s


following should be slow:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 11.6676 s, 460 MB/s

real    0m11.670s
user    0m0.011s
sys 0m1.989s

following should be fast:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 0.810326 s, 6.6 GB/s

real    0m0.811s
user    0m0.005s
sys 0m0.806s

borg backups 8 1GB files...
Initializing repository at "repo"
Encryption NOT enabled.
Use the "--encryption=repokey|keyfile|passphrase" to enable encryption.
Synchronizing chunks cache...
Rebuilding archive collection. Known: 0 Repo: 0 Unknown: 0
Merging collection into master chunks cache...
Done.

if following is slow, backing up the 5GB file removed it from the cache:
5120+0 records in
5120+0 records out
5368709120 bytes (5.4 GB) copied, 0.827725 s, 6.5 GB/s   <-- fast

real    0m0.833s
user    0m0.005s
sys 0m0.821s


Test script used
================

#!/bin/sh
# test whether borg kills the cache
# machine has Haswell i5-4200, 8GB RAM, same SSD as src and target

#only done once:
#for I in `seq 1 8` ; do
#    dd if=/dev/zero of=1GB_$I bs=1M count=1024
#done
#dd if=/dev/zero of=5GB bs=1M count=5120

sync
sudo bash -c 'echo 3 > /proc/sys/vm/drop_caches'
sync

echo "following should be slow:"
time dd if=5GB of=/dev/null bs=1M

echo "following should be fast:"
time dd if=5GB of=/dev/null bs=1M

echo "borg backups 8 1GB files..."
rm -rf repo 2> /dev/null
borg init repo
borg create --compression zlib,6 repo::archive 1GB_*

echo "if following is slow, borg killed the cache:"
time dd if=5GB of=/dev/null bs=1M

sync
sudo bash -c 'echo 3 > /proc/sys/vm/drop_caches'
sync

echo "following should be slow:"
time dd if=5GB of=/dev/null bs=1M

echo "following should be fast:"
time dd if=5GB of=/dev/null bs=1M

echo "borg backups 8 1GB files..."
rm -rf repo 2> /dev/null
borg init repo
borg create --compression zlib,6 repo::archive 1GB_* 5GB

echo "if following is slow, backing up the 5GB file removed it from the cache:"
time dd if=5GB of=/dev/null bs=1M


Patch for borg that removes fadvise DONTNEED
============================================

diff --git a/borg/_chunker.c b/borg/_chunker.c
index 4db21b7..07e5cc9 100644
--- a/borg/_chunker.c
+++ b/borg/_chunker.c
@@ -157,7 +157,7 @@ chunker_fill(Chunker *c)
         // complete cache with data that we only read once and (due to cache
         // size limit) kick out data from the cache that might be still useful
         // for the OS or other processes.
-        posix_fadvise(c->fh, (off_t) 0, (off_t) 0, POSIX_FADV_DONTNEED);
+        //posix_fadvise(c->fh, (off_t) 0, (off_t) 0, POSIX_FADV_DONTNEED);
         #endif
     }
     else {
diff --git a/borg/repository.py b/borg/repository.py
index 559a87d..be68b69 100755
--- a/borg/repository.py
+++ b/borg/repository.py
@@ -630,6 +630,6 @@ class LoggedIO:
             if hasattr(os, 'posix_fadvise'):  # python >= 3.3, only on UNIX
                 # tell the OS that it does not need to cache what we just wrote,
                 # avoids spoiling the cache for the OS and other processes.
-                os.posix_fadvise(self._write_fd.fileno(), 0, 0, os.POSIX_FADV_DONTNEED)
+                pass # os.posix_fadvise(self._write_fd.fileno(), 0, 0, os.POSIX_FADV_DONTNEED)
             self._write_fd.close()
             self._write_fd = None