Hi, yes you can.
In edit mode, click within your table on the row that you want to turn into the heading row.
Click on "Heading row" in the now displayed edit table toolbar. The same works for a heading column. Click within a column that you'd like to make into a heading column and click on "Heading column".
You can watch how to do this in our Video #14 of the Video Tutorial Series "Learn Atlassian Confluence" and check out more about tables in Atlassians documentation on this page.