It could be a washer or O ring that needs replacing.
If it is leaking from the handle of the cold water it is most likely a worn or leaking gasket
(o-ring). I would replace that on both handles. Take a picture to HD or Lowes and show them your set up, they can guide you on which repair items you need.
You may have a washer which is not in correctly or need one.
Definitely a washer or O ring needs to be replaced. The other thing you might do is clean out your screen on the faucet. It may have pebbles if you have hard water.
Perhaps, the bottom washer for the cold handle is fine, but the top washer is worn, depending on the set up for your particular fixture. It's hard to know without taking it apart.
Sounds like it's time to take the faucet apart and change out all the rubber O-rings. Over time the rings do not seal completely. You can find instructional videos on youtube.com after you've found who the manufacturer of your faucet is. A plumbing supply store can find a replacement for you, especially if you remove the O-rings and take it to the supply with you.
OR rather than video, here's an instructional link.
This might be helpful on fixing it: https://www.youtube.com/watch?v=7OIjD9zXWDI
The cold water cartridge could have a weird worn down spot on it that is causing the leak. I'd try the replacement like you were thinking to see if it resolves the issue.
Hi Diana, hope this helps you out,