Citrus has a acid called limoene or something (apples are fine) and it is poisonous to worms.
Yesterday, while doing yard work, I moved a half rotten wooden pot thing and saw tons of red worms under it, taking advangttage of the cover and rotting wood and it was moist. Guess what? Most of them had the green reflection!