Each instrument has it's own difficulties and techniques to master. They are all equally "easy" and "hard", it all depends on the individual and how much time he/she puts into practicing the instrument.
I'll speak personally on this subject. I have played on just about every instrument there is at least once, except for the brass instruments (trumpet, tuba, etc.) & organ, but dat's coming this semester. The hardest instrument for me to play was the flute. If u don't blow in the hole just right, u won't get a sound, PERIOD.
The next hardest instrument was the violin. If u got fat fingers, u gonna struggle cuz the fret board is like 1/16 the size of a guitar or bass guitar. Plus, there are no lines or frets, so u betta know what note(s) u hitting. Not to mention da dern bow, there is a certain way to hold it and strum it across the strings.
The easiest for me to learn was the clarinet. It took a while to get a consistent sound, but the fingerings were easy. As it is with the flute and the other woodwind instruments, breath control will be the hardest skill to master. Too much breath and u will get a high-pitched squeak. Too little breath and u will get little to no sound, and u may pass out cuz u running out of air.
The other instruments were in the middle, not good, but not bad either. The great thing about instruments other than the piano, organ, guitar, and drum stuff is 99% of the time u playing one note at a time. Scales, fingering, and breathing exercises will help you to master them.