The prison for criminal wizards and witches, Azkaban, is first introduced in Harry Potter and the Chamber of Secrets and takes center stage in Harry Potter and the Prisoner of Azkaban.
The name itself is interesting because of its strong association with Alcatraz, the former prison located on Alcatraz Island in San Francisco. They are similar in that they are both located on islands, were reputed to be escape-proof, and were reserved for those who committed the most heinous of crimes.
The name Azkaban may also have been inspired by Arthur Conan Doyle's The Hound of the Baskervilles, a story that features escaped convicts and large, menacing dogs. Interestingly, the Grim in Prisoner of Azkaban, a dog that portends death, may have been inspired by the location of the prison in Conan Doyle's story: Grimpen Mire.
Azkaban also bears similarities to Devil's Island, made famous in the film Papillon, and perhaps Angband, the name of the fortress of Morgoth in J.R.R. Tolkien's works.
The peculiar thing about Azkaban, from a technical perspective, is that it is actually a poorly defended and relatively insecure installation. By the time the Harry Potter novels end, dozens if not hundreds of people have escaped. Sirius Black manages it in Prisoner of Azkaban through trickery that was so obvious that others should have managed it sooner. Barty Crouch Jr. is helped out by his bereaved family in Goblet of Fire. And when the Dementors, the guardians of Azkaban, rebel, the prison becomes essentially useless. Rowling apparently did not consider the old adage that asks: "who watches the watchmen?" except when Dumbledore expresses his concerns about the Dementors.
The name remains the same in the translations of the Harry Potter novels. Even the French version, in which names are often changed, Azkaban is Azkaban. Even the Latin version of the novels preserves the name, including its spelling, which given that Latin had no "k" until the Middle Ages, is clearly a choice aimed at modern readers of Latin and not the language as Caesar or Cicero knew it.
Unlike many of the other names in the Harry Potter novels, including Voldemort and Malfoy, Azkaban does not bear particularly sinister overtones by itself. It is through the characters, starting with Rubeus Hagrid in Harry Potter and the Chamber of Secrets, that we the readers learn to fear the place and its effects on prisoners there. It is located in a remote, secret site in the North Sea, guarded by the monstrous Dementors, and reputed to drive all who go there insane.
Perhaps then the choice of a relatively neutral name for this prison was excellent. When first mentioned in Chamber of Secrets, we know little of what to expect about it. Compared to the Forbidden Forest, its name says nothing about what kinds of prisoners are housed there or under what conditions. Like Harry himself, we find out one step at a time that a sentence at Azkaban is a fate almost worse than death. Harry fears being sent there after he inflates his aunt at the start of The Prisoner of Azkaban, and by the end of that book knows all too well the effects the prison has on its convicts.