It took me accessing the webpage for the Archives of Diocese of Hartford to find a good explanation of how to trace Episcopal church records in Connecticut. First and foremost, they do not function like Catholic records. Unless the church has closed, the Diocese does not hold copies of the baptismal, death or marriage records. According to their page, you need to contact the parish or the Family History Library.
I turned to the FamilySearch.org catalog to test their theory, using Church of the Epiphany (Durham). The Family History Library does have a microfilm of parish records, but the reel ends in 1940. After that, you'll need to contact the parish. And the index page had one important note: it was created from originals at the Connecticut State Library. So, as it turns out, you have three options: the parish, the State Library, and the Family History Library. I always start with the parish.