The Residency, now housing the Simonstown Museum, is on the main road, between the naval dockyard and the Anglican church. The house was originally built as a residence for Dutch governors, on their occasional visits to the port, but it only served that purpose for a few years. When the Royal Navy occupied Simonstown in 1814, it became the seat of the Government Resident, or magistrate, and remained so until 1980.
Some of the doors in the Residency were once cabin doors on old sailing ships. There are stocks where prisoners were restrained for humiliating punishments, and cells where men were chained. Some say that men were tortured here, but apart from the use of the whip or cat, this would have been illegal during the British occupation (which does not necessarily mean that it didn't happen.) In one cell, there are bloodstains and, sometimes, two miserable phantom prisoners to be seen. The spectre of an ancient mariner, unjustly flogged to death, has been seen in the building, as has the grim ghost of a warder's wife, who in life had abused female prisoners.
Photos of a mural in the Residency are often, for no apparent reason, blurred or blank.
In the building is a bar room, originally for visiting sailors, and on the wall hangs the portrait of a young gentlewoman. The ghost of this woman is another "grey lady". Margaret Williamson, who has done the best research on the Simonstown ghosts, wrote that she is sometimes called the "lavender lady", or "lilac lady". She was often seen by the wives of magistrates, and some of the magistrates themselves admitted that the ghost exists. One magistrate, Duncan Neethling, saw the grey lady following his wife around the kitchen. The ghost has also been encountered since the Residency became a museum. Usually friendly, she once surprised a member of the staff by slamming a door on his fingers. Unlike most spectres, this one has been known to speak. She has been seen both inside and outside of the building.
Margaret Williamson convincingly argues that the ghost is not that of the teenaged Eleanor Macartney, daughter of the first British governor. She also argues that the spirit is not that of a young woman who loved Horatio Nelson, but in this she may be mistaken.
In 1776, Nelson came ashore from his ship, The Dolphin, to be nursed through an illness. This was long before the Cape became a British possession. Internet searches sometimes reveal that the Residency was built in 1777, but the authors are probably using Margaret Williamson's book, Haunted Corners, as their source. According to that, Nelson must have recuperated at the Cape before the Residency was even built.
Phillida Brooke Simons, however, in her book Cape Dutch Houses, points out that in 1776, the building had already been used by the Cape's governors for five years. In fact, that was the year in which the building ceased to be used for that purpose, and was sold to Gideon Rousseau, a wealthy entrepeneur with twelve children.
The colonial Dutch were renowned for their hospitality towards polite visitors. If Nelson came ashore at Simonstown while Rousseau owned the house, it is very likely that he would have been invited to stay there, although the building which later became Admiralty House was also used to accomodate visiting seamen.
In any case, as Lawrence Green pointed out, Nelson also visited Simonstown while in command of The Badger.
I suggest that the ghost is that of one of Gideon Rousseau's daughters. Whoever she is, she is said to have drowned herself when her lover, whether Nelson or somebody else, had to leave the Cape.
There were recently organised Ghost Tours starting at the museum, but I gather that they are now
in abeyance. Of course, there is nothing to stop you from going on your own ghost hunting
expedition.