I agree, he's rather a horrid character, which is why I intended him to be quite a menace to poor Bertie rather than understanding. In this story, I was planning that Baxter is in love with the character that Bertie accidentally finds himself engaged to, so Baxter tries to snoop around him (convincing Lord Emsworth further that he's insane) and find out why he's so close to Jeeves, to implicate him and get him in trouble with the law, so he can have the girl to himself.
I didn't explain it clearly as I didn't want to give too much away. :)