Is it possible to exclude payment option at product level?
Yes. All you need to do is to add one or more config variables to your module so that you can identify the products. The module looks for those products in the order and throws an exception if they are present.
how can I add config variable in product? you mean custom attribute?
he said "add one or more config variables to your module" One of these might record values of some attribute on your products (could be a custom field or attribute) that you can use for your logic.