The context menu (via right-click) on Translation Memories shows "Remove From List" right above "Delete Translation Memory". This is a dangerous place and I am sure I'm not the only one who has accidentally deleted a TM simple by moving the mouse.
The issue was brought up during last year's SDL Partner conference by a number of users, but nothing has been done about it yet.
I would suggest to put the option as far away from the "remove From List" item as possible to avoid disasters.