There is a bug in Android where FragmentPageLoader doesn't respond to notifyDataSetChanged() (see http://code.google.com/p/android/issues/detail?id=19001).
The end result is that there is a lot of weirdness when you flag an activity and then swipe back again. I'm inclined to leave this for now and see if the bug gets fixed on the Android end.