By default, the Accordion module has a plus icon when the content is collapsed and a minus icon when the content is expanded, as shown in this screenshot.
Using some custom CSS, you can change these icons to images that show the expanded and collapsed state.
To change the Accordion icons to images:
From the WordPress admin panel, click Media and upload your images to the Media Library. Make a note of your image URLs so you can insert them into the CSS.
Paste in the CSS code below and replace the URLs with the correct ones for your images. Omit the last rule if you're using the same image for both states.
See this article about where to add custom CSS code.
Click Save & Publish.