I had this happen as well. I think it was the MKV format had some kind of permission or "lock". But, I had several downloads in the queue and one of them was stuck downloading more segments than it had (like downloading 2,147,182 out of 1,889) - not sure what caused that either! Same experience as OP, using Windows Explorer I could not Move or Delete the file (when Moving - it was stuck on "Calculating..."), but I could Copy the file.
To resolve it, I had to resort to "old school" methods. Opened a Command Prompt as a System Administrator and deleted using the "del" command. This was after I copied them to my Plex folder