Use the tdev pointer directly instead of going through the port data
when accessing the serial data in close().

Signed-off-by: Johan Hovold <johan@kernel.org>