math - Strange behavior when implementing Back propagation in DBN -


Currently I'm trying to implement Deep Trust Network. But I have met a very strange problem. My source code can be found here:

I first applied RBM using CD and it works perfectly (using Gong's concurrency feature is very fast). Then I start implementing a normal feed forward network with diffusion and then it is strange. It looks very unstable when I run it with the Ether gate test, it sometimes fails, when I set the hidden layer nodes of 10 or more, it never fails.

Step 2: Calculate all activations with bias

Step 2: Calculate the output error

Step 3: Promote the back error of each node

Step 4: Calculate Delta weight and bias with speed for each node

Step 1 to step 4 I make a full batch and these delta weight and bias Yoga <

Step 5: Apply on the average delta weight and bias

I have found the tutorial

And this usually works if I give it more hidden layer nodes. My test code is here

So I think this should work and RMB and DBN should be implemented by adding normal NN. However, the result gets very bad. It can not even learn an Xor gate in 1000 travels. And sometimes it goes completely wrong. I tried to debug it, then I rebuilt a DBN after Pretren. Most of the time rebuilding looks good, but the previous campaign fails even when the pretren result is correct.

I do not really know what is wrong with the previous publicity. I should be misunderstood the algorithm or make some major mistakes in the implementation.

If possible, please run the test code and you will see how strange it is. The code itself is quite readable, any sign will be very helpful. Thank you in advance

I remember that you are saying Hinton that you do not train RMB on XOR There is something about the vector space, which does not allow two layer networks to work. Deep networks have fewer linear properties which allow it to work.

Comments

Post a Comment

Popular posts from this blog

Java - Error: no suitable method found for add(int, java.lang.String) -

I'm in the middle of homework work and I'm stuck. At this point in my code, I think that I should have a GUI window that opens and allows me to type "inserted text number". Notice is not going anywhere but at this point, If I pass through the problem then it will be in a linked list. I am getting two of the same error for the lines. Add (index, element); And I can not seem to get past it, there is no suitable method for the error "add (int, java. string string)". Code is below, please advise. To clarify - this will not be a method error because it is a linked list. There should not be any way involved. import java.awt. *; Import java.awt.event. *; Import javax.swing. *; Import java.util. *; Import java.util.Scanner; Import java.util.LinkedList; Public class TopTenList JFrame {Private TopTenList tt; See the Private JTextArea list; Private JTextField cmdTextField; Private JTextField resultsTestfield; // This GUI window is the code for the public toptist...
Read more

java - JPA TypedQuery: Parameter value element did not match expected type -

I am using JPA 2.0 and getting the following code in the DAO layer: Public Zero Test () {string key = "status"; String [] Conditions = {"A", "B"}; & Lt; TestTable & gt; Results = Search (Keys, Conditions); } Public listing & lt; TestTable & gt; Search (string key, object value error) {string sql = "test ndf to testlet and jade nand." + Key + "in:" + key; TypedQuery & LT; TestTable & gt; Query = entityManager.createQuery (SQL, TestTable.class); Query.setParameter (key, Arrays.asList (valueArr)) / *** Error Line *** Return query.getResultList (); } In the above error line, it throws the following exception: java.lang.IllegalArgumentException: parameter value element [[Ljava.lang.String; @ cbe5bc] did not match the expected type [java.lang.String] Why is this expected type string while actually this string []? Please help! Note: This is derived from the normal routine and is a simplified code. I can no...
Read more

c++ - static template member variable has internal linkage but is not defined -

Yes, I know, there is a question with almost the same title, but it refers to a different situation. Error message) In my case, I have a .cpp file with a big named designation name (implementation details). There is a property class template with a static data member in that name space, which I need to access from outside the unknown namespace. I give it a little bit of meat: file.hpp namespace Bar {template & lt; Typename A & gt; Struct foo {static_assert (is_same & lt; a, float & gt; :: value} is_same & lt; a, double>: value, ""); Fixed zero set_static_var (a const & x); // ...}; } and file.cpp namespace {template & lt; Typename A & gt; Struct foo_traits {// supports the implementation of several static code bars: foo & lt; & Gt; Fixed A data; }; The template's & lt; & Gt; Float foo_traits & lt; Float & gt; :: datum; // No change if this is in the global namespace template & lt; & Gt; Doub...
Read more