You can optionally catch the largest Fish at the Pony, the Hylia Loach. If you catch a fish that is thirteen pounds or more, show it to the man you’ll win the Golden Scale. Just keep at it using this method and you’ll eventually catch a big fish.
The other piece of advice I’d give is DON’T SCARE THE FISH! If you do, they will swim to the deepest part of the pond and you won’t be able to catch them. It can be frustrating if the Fish get away, but don’t worry, it’s not just you.