They're both "th" sounds, the ð is like the "th" in "weather" in English whereas the þ is like the "th" in "think", which are slightly different sounds in English and also in Icelandic.
Put another way, ð is "voiceless alveolar non-sibilant fricative" whereas þ is "voiced alveolar non-sibilant fricative" which is to say that the difference is that one of them is "voiced" whereas one isn't, in other word the voicebox vibrates for one of them and not for the other ..... not sure that helps though ....
In general, if English speakers treat them both as "th" then all will be well, the position of the sound in the word will make enough of a difference without having to think about it.