Hi meme,
This sounds suspicious, especially if it is happening with a second mouse. And since even the click is being affected not just the position of the pointer this rules out any surface issues.
I recommend performing a windows reset, this will retain your data and apps but it will restore Windows to default settings. This could very well be a hardware issue, overheating comes to mind. If the computer is older then 4 years, it may very well be time to upgrade anyway.