Aunt May tells a young girl Kate the story of how her brother (the Boy) once met a Borrower. Arrietty is a Borrower who lives under the floor of a kitchen in an old house in the country. One day her father, Pod, is seen climbing down a curtain by a boy who they didn't realize was living there. The Boy helps Pod down with the teacup (from a dollhouse) he was carrying. When he tells his wife Homily about what happened, she becomes nervous that they must leave the house.

Pod and Homily tell Arrietty the next day about how he was spotted, and tell a story about how there used to be other Borrowers who lived in the house. One (her uncle Hendreary) was spotted, and they had to leave after a cat (presumably) killed Arrietty's cousin Eggletina. Pod begins to take Arrietty out to Borrow to help her learn the importance of not being spotted (something Eggletina was never taught).

The first day she leaves her house under the kitchen she runs into the Boy, and they become friends. She continues to visit him and read to him until he discovers their hiding spot. He doesn't tell anyone, however, and begins to bring the Clock family various items. Homily loves all the new possessions.

The Housekeeper Mrs. Driver begins to notice items going missing, so she stays up one night to find out what is happening. She hears the Boy, but doesn't catch him giving items to the Borrowers. She is able to discover their hiding spot, but they disappear before she can show them to anyone else. She calls in the police and the exterminators, but no other evidence of them is found.

The book ends with an implication that the Boy eventually found some evidence the Clocks found their way to the Badger hole where their uncle lived. Aunt May implies at the end, however, that all of it could have been fabricated by her brother.