In order to use file extensions in playlists, they must be visible in your library. Open Usher's preferences, click the Appearance icon, and make sure 'Show file name suffixes' is checked.

Once that's done, create a new Smart Playlist, and set its criteria to (at least) Name - Ends With, and then specify the extension bit (.wmv, for instance).