Studying audience first often makes sense for a business, as suggested above. I'm going to start in the other end however, and assume you've already created the product (or a large part of it) and are now looking to find out what the demand is for it.
There's many sources for gathering Market insights and data, here's some examples:
-Buy specific market data from research institutes, or commission a specific study from them. Expensive for a small business but probably most comprehensive.
-Find out if there's an industry association of some form. They often aggregate surprisingly insightful data on how their member companies are doing, market growth trends and such.
-Study competitors or similar offerings. Do they appear successful? Large orgs often publish results. Which products seem to sell well? Do they have a best sellers list, or way to filter assortment by popularity? If not, consider calling them as a regular customer who is curious about what is popular.
-Use free online tools like Google Trends (quantitative) or Answer the Public (qualitative) to see which search topics are of interest to people, and if those topics are growing or not. Search data is generally a great indicator for overall interest in something.
-Find your target group and talk to them. If the course is for a subcommunity, e.g. Scuba Divers, probably there is already an online forum (reddit), social media group or other place they congregate. Find that, review the conversations already happening and consider asking. If you can involve the community early on in your creation you will probably get both useful feedback and a lot of goodwill.
-Building on the above, make your work partially public as you go and gauge interest/learn from that. An increasingly popular way to write a book is to break your material up into blog posts. Those who perform best become part of the book, maybe even expanded upon, others may be rewritten based on feedback or cut in the process.
-Bonus tip: Learn from other geographies. Is something trending in another market where data (any of the above ideas) is more accessible? Is it reasonable to expect this trend/interest will be exported?
Good luck! Jacob