Do I need to set up payees for each checking account?

No, you may use the same payees for multiple checking accounts that you have displayed on your online banking for which you are an authorized signor.  When processing your payment you can select the appropriate "Pay From Account".  The money will then be debited from that specific account.