If the computed reply size is negative, something went wrong, treat it
as an error.

v2: Be more careful about size_t being unsigned (Matthieu Herrb)
v3: SIZE_MAX not SIZE_T_MAX (Alan Coopersmith)
v4: backport to nx-libs 3.6.x (Mike DePaulo)
Reviewed-by: Julien Cristau <jcristau@debian.org>
Reviewed-by: Michal Srb <msrb@suse.com>
Reviewed-by: Andy Ritger <aritger@nvidia.com>
Signed-off-by: Adam Jackson <ajax@redhat.com>
Signed-off-by: Alan Coopersmith <alan.coopersmith@oracle.com>
Signed-off-by: Fedora X Ninjas <x@fedoraproject.org>
Signed-off-by: Dave Airlie <airlied@redhat.com>