This happens in RhinoNest 2.1.0.1 and RhinoNest for Grasshopper 0.2. In Rhino 4 and 5.
RhinoNest shows a memory error. After I say OK, the nest can be run again but RhinoNest does not generate multiple sheets.
Sometimes Rhino simply crashes away.
The Task Manager shows plenty of memory still available...